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Preface 

Industrial robots have been widely applied in many fields to increase productivity 
and flexibility, i.e. to work on repetitive, physically tough and dangerous tasks. Be- 
cause of similar reasons, the need on robots in service sectors-like robots in the 
hospital, in household, in underwater applications-is increasing rapidly. Mobile, 
intelligent robots became more and more important for science as well as for in- 
dustry. They are and will be used for new application areas. 

The range of potential applications for mobile robots is enormous. It includes agri- 
cultural robotics applications, routine material transport in factories, warehouses, 
office buildings and hospitals, indoor and outdoor security patrols, inventory veri- 
fication, hazardous material handling, hazardous site cleanup, underwater applica- 
tions, and numerous military applications. 

This book is the result of inspirations and contributions from many researchers 
worldwide. It presents a collection of wide range research results of robotics scien- 
tific community. Various aspects of current research in new robotics research areas 
and disciplines are explored and discussed. It is divided in three main parts cover- 
ing different research areas: 

Humanoid Robots, 

Human-Robot Interaction, 

Special Applications. 

I hope that you will find a lot of useful information in this book, which will help 
you in performing your research or fire your interests to start performing research 
in some of the cutting edge research fields mentioned in the book. 



Editor 
Aleksandar Lazinica 
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Humanoid Robot Navigation Based on Groping 
Locomotion Algorithm to Avoid an Obstacle 

Hanafiah Yussof 1 , Mitsuhiro Yamano 2 , Yasuo Nasu 2 , Masahiro Ohka 1 

1 Graduate School of Information Science, Nagoya University 
2 F acuity of Engineering, Yamagata University 

Japan 

1. Introduction 

A humanoid robot is a robot with an overall appearance based on that of the human body 
(Hirai et aL, 1998, Hirukawa et aL, 2004). Humanoid robots are created to imitate some of 
the same physical and mental tasks that humans undergo daily. They are suitable to coexist 
with human in built-for-human environment because of their anthropomorphism, human 
friendly design and applicability of locomotion (Kaneko et aL, 2002). The goal is that one 
day humanoid robots will be able to both understand human intelligence and reason and act 
like humans. If humanoids are able to do so, they could eventually coexist and work 
alongside humans and could act as proxies for humans to do dangerous or dirty work that 
would not be done by humans if there is a choice, hence providing humans with more 
safety, freedom and time. 

Bearing in mind that such robots will be increasingly more engaged in human's 
environment, it is expected that the problem of "working coexistence 7 ' of humans and 
humanoid robots will become acute in the near future. However, the fact that no significant 
rearrangment of the human's environment to accomodate the presence of humanoids can be 
expected. Eventually, the "working coexistence" of humans and robots sharing common 
workspaces will impose on robots with their mechanical-control structure at least two 
classes of tasks: motion in a specific environment with obstacles, and manipulating various 
objects from the human's environment (Vukobratovic et aL, 2005). As far as this working 
coexistence is concerned, a suitable navigation system combining design, sensing elements, 
planning and control embedded in a single integrated system is necessary so that humanoid 
robots can further "adapt" to the environment previously dedicated only to humans. To 
date, research on humanoid robots has arrived at a point where the construction and 
stabilization of this type of robot seems to be no longer the key issue. At this stage, it is 
novel practical applications such as autonomous robot navigation (Saera & Schmidt, 2004, 
Tu & Baltes, 2006), telerobotics (Sian et aL, 2002) and development of intelligent sensor 
devices (Omata et aL, 2004) that are being studied and attracting great interest. Autonomous 
navigation of walking robots requires that three main tasks be solved: self -localization, 
obstacle avoidance, and object handling (Clerentin et aL, 2005). In current research, we 
proposed a basic contact interaction-based navigation system called "groping locomotion" 
on the humanoid robots capable of defining self-localization and obstacle avoidance. This 
system is based on contact interaction with the aim of creating suitable algorithms for 
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humanoid robots to effectively operate in real environments. In order to make humanoid 
robot recognize its surrounding, six-axis force sensors were attached at both robotic arms as 
end effectors for force control. 
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Fig. 1. Robot locomotion in the proposed autonomous navigation system. 

Figure 1 explains the phylosophy of groping locomotion method on bipedal humanoid 
robot to satisfy tasks in autonomous navigation. Referring to this figure, the humanoid robot 
perform self-localization by groping a wall surface, then respond by correcting its 
orientation and locomotion direction. During groping locomotion, however, the existence of 
obstacles along the correction area creates the possibility of collisions. Therefore, the 
humanoid robot recognize the existance of obstacle in the correction area and perform 
obstacle avoidance to avoid the obstacle. 

Some studies on robotics have led to the proposal of an obstacle avoidance method 
employing non-contact interaction, such as vision navigation and image processing (Seydou 
et al., 2002, Saera & Schmidt, 2004), while others use armed mobile robots and humanoids 
on a static platform (Borenstein & Koren, 1991). There has been very little research reported 
about the application of a contact interaction method to avoid obstacles in anthropomorphic 
biped humanoid robots. In this report, we focus on a development of an autonomous 
system to avoid obstacles in groping locomotion by applying multi-tasking algorithm on a 
bipedal 21-DOF (degrees-of-freedom) humanoid robot Bonten-Maru II. Consiquently, we 
presents previously developed bipedal humanoid robot Bonten-Maru II that used in the 
experiments and evaluations of this research project. In addition, we explain the overall 
structure of groping locomotion method and its contribution in the humanoid robot's 
navigation system. We also explain simplified formulations to define trajectory generation 
for 3-DOF arms and 6-DOF legs of Bonten-Maru II. Furthermore, this report includes an 
experimental results of the proposed obstacle avoidance method using Bonten-Maru II that 
were conducted in conjunction with the groping locomotion experiments. 



2. Relevancy of Contact Interaction in Humanoid Robot's Navigation 

Application of humanoid robots in the same workspace with humans inevitably results in 
contact interaction. Our survey on journals and technical papers resulted to very small 
number of work reported about the application of a contact interaction method to navigate 
humanoid robots in real environments. Some studies in robotics have proposed methods of 
interaction with environments using non-contact interaction such as using ultrasonic wave 
sensor, vision image processing and etc (Ogata et al., 2000, Cheng et al., 2001). However, 
some work reported the use of robotic armed mobile robot to analyze object surface by 
groping and obtain information to perform certain locomotion (Hashimoto et al., 1997, 
Kanda at al., 2002, Osswald et al., 2004). Overall there has been very little work reported 
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about application of contact interaction on bipedal humanoid robots (Konno, 1999). 
Eventually, most report in navigation of walking robot is related to perception-guided 
navigation (Clerentin et al., 2005), particularly related to visual-based navigation that has 
been a relevant topic for decades. In visual-based navigation, which is classified as non- 
contact interaction, besides the rapid growth in visual sensor technology and image 
processing technology, identification accuracy problems due to approximate data obtained 
by the visual sensor and interruption of environment factors such as darkness, smoke, dust, 
etc. seems to reduce the robots performances in real environments. 

Meanwhile, contact interaction offers better options for humanoid robots to accurately 
recognize and structure their environment (Coelho et al., 2001, Kim et al., 2004), 
making it easier for them to perform tasks and improve efficiency to operate in real 
environment. We believe that contact interaction is a relevant topic in research and 
development of humanoid robot's navigation. Indeed contact interaction is a 
fundamental feature of any physical manipulation system and the philosophy to 
establish working coexistence between human and robot. 

3. Definition of Groping Locomotion 

Groping is a process in which the humanoid robot keeps its arm in contact with the wall's 
surface while performing a rubbing-like motion. The proposed groping locomotion method 
comprises a basic contact interaction method for the humanoid robot to recognize its 
surroundings and define self -localization by touching and groping a wall's surface to obtain 
wall orientation (Hanafiah et al., 2005a, Hanafiah et al, 2005b). Figure 2 shows photograph of 
the robot and robot's arm during groping on the wall surface. During groping process, 
position data of the end effector are defined, which described the wall's surface orientation. 
Based on the wall's orientation, relative relations of distance and angle between the robot and 
the wall are obtained. The robot then responds to its surroundings by performing corrections 
to its orientation and locomotion direction. Basically, the application of sensors is necessary for 
a humanoid robot to recognize its surroundings. In this research, six-axis force sensors were 
attached to both arms as end effectors that directly touch and grasp objects and provide force 
data that are subsequently converted to position data by the robot's control system. 




Fig. 2. Photographs of robot and robot's arm during groping on wall surface. 



In this research, the groping process is classified into two situations: groping the front wall and 
groping the right-side wall. Figures 3(a) and (b) shows plotted data of the end effector position 
during groping front wall and right-side wall, which described the wall surface orientation that 
positioned at the robot's front and right side, respectively. The end effector data obtained during 
groping process are calculated with the least-square method to define wall's orientation. Based on 
the wall's orientation obtained in groping process, the relative relation of humanoid robot's 
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position and angle are defined, like shown in Fig. 4. Here, (j) is groping angle , and 90° - (/) is a 
correction angle. Meanwhile L is the shortest distance from the humanoid robot to the wall. 
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(a) Groping front 
Fig. 3. Graph of end effector position in groping locomotion. 
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(b) Groping right-side 




Fig. 4. Robot orientation after groping wall. 



4. Obstacle Avoidance in Groping Locomotion Method 

4.1 Definision of Obstacle Avoidance in Humanoid Robot Navigation System 

In humanoid robot navigation, abilities to recognize and avoid obstacles are inevitably 
important tasks. The obstacle avoidance method proposed in this research is a means to 
recognize and avoid obstacles that exist within the correction area of groping locomotion by 
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applying a suitable algorithm to the humanoid robot's control system. The proposed 
obstacle avoidance algorithm is applied to a bipedal humanoid robot whose arms were 
equipped with six-axis force sensors functioned to recognize physically the presence of 
obstacles, then ganerate suitable trajectory to avoid it. 



4.2 Groping Locomotion Algorithm 

In the groping-locomotion method, an algorithm in the humanoid robot's control system controls 
the motions of the robot's arms and legs based on information obtained from groping process. 
The algorithm comprises kinematics formulations to generate trajectory for each robotic joint. 
The formulations involve solutions to forward and inverse kinematics problems, and 
interpolation of the manipulator's end effector. It also consists of force-position control 
formulations to define self-localizasion of the humanoids body based on force data that obtained 
in groping process. Figure 5 shows a flowchart of the groping locomotion algorithm. Basically, 
the algorithm consists of three important processes: searching for a wall, groping a wall's surface, 
and correction of robot position and orientation. The algorithm is applied within the humanoid 
robot control system. Figure 6 displays the control system structure consists of two main process 
to control the humanoid robot motion: robot controller and motion instructor. Shared memory is 
used for connection between the two processes to send and receive commands. The motion 
instructor, also known as user controller, initially check whether instruction from robot controller 
has an access permission or not before motion instructor sending request motion commands to 
perform motion. The command requested by motion instructer is send to shared memory and 
transfer to robot controller. Based on groping locomotion algorithm, the robot controller generate 
nacessary trajectory and send its commands to humanoid robot's joints in order to perform 
required motion. Lastly, when the motion is completed, new access permission will send to 
motion instructor for delivery of the next instruction commands. 
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Fig. 6. Control system structure of humanoid robot Bonten-Maru II. 

4.3 Correlation of Obstacle Avoidance with Groping Locomotion Algorithm 

Research on groping locomotion has led to the proposal of a basic contact interaction 
method in humanoid robot's navigation system. In groping locomotion, a robot's arm 
gropes a wall surface to obtain the wall's orientation data by keeping its arm in contact with 
the wall's surface, and corrects its position and orientation to become parallel with the wall. 
Here, the proposed obstacle avoidance method is designed to avoid obstacles existing at the 
correction area. Figure 7(a) shows flowchart of the obstacle avoidance algorithm. The 
algorithm consists of three important processes: checking the obstacle to the left, rotating 
toward the back-left position, and confirming the obstacle's presence. The algorithm is 
based on trajectory generation of the humanoid robot's legs, with reference to the groping 
results in groping locomotion. Meanwhile, Fig. 7(b) shows the flowchart of groping 
locomotion algorithm combined with the proposed obstacle avoidance algorithm. The 
combined algorithm is complied in the robot's control system, as described in Fig. 6, to 
perform tasks in humanoid robot's navigation system. 



4.4 Analysis of Obstacle Avoidance Algorithm 

The concept of the proposed obstacle-avoidance algorithm is based on trajectory generation of 
the humanoid robot's legs, with reference to the groping results. Leg positions are decided by 
interpolation using polynomial equations, and each leg-joint position is given via angle data 
from calculation of the inverse kinematics needed to move the legs to the desired positions. 
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(a) Obstacle avoidance algorithm. 

Fig. 7. Application of obstacle avoidance 



(b) Groping locomotion algorithm combined with 
obstacle avoidance algorithm, 
algorithm to groping locomotion algorithm. 



Basically, obstacle avoidance is performed after correcting the robot's distance to the wall, before 
proceeding to the correct angle. While checking the obstacle to the left, the left arm will search for 
and detect any obstacle that exists within the correction angle's area and up to the arm's 
maximum length in order to instruct the robot's system either to proceed with the correction or to 
proceed with the next process of obstacle avoidance. If an obstacle is detected, the robot will rotate 
to the back-left position, changing its orientation to face the obstacle. The robot will then 
continuously recheck the existence of the obstacle by performing the " confirm obstacle" process. If 
no obstacle is detected, the robot will walk forward. However, if an obstacle was detected, instead 
of walking to forward direction, the robot will walk side-step towards its left side direction, and 
repeat again the confirmation process until no obstacle is detected. The robot will then walks 
forward and complete the obstacle avoidance process. 



4.4.1 Checking for Obstacles to the Left 

While checking for an obstacle, if the arm's end effector touches an object, the force sensor will 
detect the force and send the force data to the robot's control system. Once the detected force 
exceeds the parameter value of maximum force, motion will stop. At this moment, each encoder at 
the arm's joints will record angle data and send them to the robot control system. By solving the 
direct kinematics calculation of the joint angles, the end effector's position is obtained. The left 
arm's range of motion while checking for obstacles is equal to the correction angle, 90° - (j), where <p 
is the groping angle. Any objects detected within this range are considered as obstacles. 

4.4.2 Rotate to Back-Left Position 

Once an obstacle has been detected during the process of checking for an obstacle to the left, 
the robot will rotate its orientation to the back-left position "facing" the obstacle in order to 
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confirm the obstacle's position at a wider, more favorable angle, finally avoiding it. At first, 
the left leg's hip-joint yaw will rotate counterclockwise direction to 90° - (p. At the same 
time, the left leg performs an ellipse trajectory at Z-axis direction to move the leg one step 
backward to a position defined at X-Y axes plane. At this moment the right leg acts as the 
support axis. The left leg's position is defined by interpolation of the leg's end point from its 
initial position with respect to the negative X-axis position and positive Y-axis position of 
the reference coordinate at a certain calculated distance. Then, the robot corrects its 
orientation by changing the support axis to the left leg and reverses the rotation clockwise of 
the left leg's hip-joint yaw direction of the angle 90° - <f). Finally, the robot's orientation is 
corrected to "face" the obstacle. 

4.4.3 Confirm Obstacle 

After the obstacle is detected and the robot orientation has changed to face the obstacle, it is 
necessary to confirm whether the obstacle still exists within the locomotion area. This 
process is performed by the robot's right arm, which searches for any obstacle in front of the 
robot within its reach. If the obstacle is detected within the search area, the arm will stop 
moving, and the robot will perform side-step to left direction. The robot's right arm will 
repeat the process of confirming the obstacle's presence until the obstacle is no longer 
detected. Once this happens, the robot will walk forward in a straight trajectory. These steps 
complete the process of avoiding the obstacle. 

5. Application of Groping Locomotion Method in Humanoid Robot Navigation 
System 

The development of navigation system for humanoid robots so that they can coexist and 
interact with humans and their surroundings, and are able to make decisions based on their 
own judgments, will be a crucial part of making them a commercial success. In this research, 
we proposed a basic navigation system called "groping locomotion" on a 21-DOF humanoid 
robot Bonten-Maru II. The groping locomotion method consists of algorithms to define self- 
localization and obstacle avoidance for bipedal humanoid robot. This system is based on 
contact interaction with the aim of creating suitable algorithms for humanoid robots to 
effectively operate in real environments. 

5.1 Humanoid Robot Bonten-Maru II 

In this research, we have previously developed a 21-DOF (degrees-of-freedom), 1.25-m tall, 
32.5-kg anthropomorphic prototype humanoid robot called Bonten-Maru II. The Bonten-Maru 
II was designed to mimic human characteristics as closely as possible, especially in relation 
to basic physical structure through the design and configuration of joints and links. The 
robot has a total of 21 DOFs: six for each leg, three for each arm, one for the waist, and two 
for the head. The high numbers of DOFs provide the Bonten-Maru II with the possibility of 
realizing complex motions. Figure 8 shows a photograph of Bonten-Maru II, the 
configuration of its DOFs, and physical structure design. 

The configuration of joints in Bonten-Maru II that closely resemble those of humans provides 
the advantages for the humanoid robot to attain human-like motion. Each joint features a 
relatively wide range of rotation angles, shown in Table 1, particularly for the hip yaw of 
both legs, which permits the legs to rotate through wide angles when avoiding obstacles. 
Each joint is driven by a DC servomotor with a rotary encoder and a harmonic drive- 
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reduction system, and is controlled by a PC with the Linux OS. The motor driver, PC, and 
power supply are placed outside the robot. 
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Fig. 8. Humanoid robot Bonten-Maru II and configuration of DOFs and joints. 



Axis 


Bonten-Maru II (deg) 


Neck (roll and pitch) 


-90 ~ 90 


Shoulder (pitch) right & left 


-180 ~ 120 


Shoulder (roll) right/ left 


-135~30/-30~135 


Elbow (roll) right/ left 


~ 135/0 ~ -135 


Waist (yaw) 


-90 ~ 90 


Hip (yaw) right/ left 


-90 ~ 60/-60 ~ 90 


Hip (roll) right/ left 


-90-22/-22-90 


Hip (pitch) right & left 


-130 ~ 45 


Knee (pitch) right & left 


-20 -150 


Ankle (pitch) right & left 


-90 ~ 60 


Ankle (roll) right/ left 


-20-90/-90-20 



Table 1. Joint rotation angle. 

In current research, Bonten-Maru II is equipped with a six-axis force sensor in both arms. As 
for the legs, there are four pressure sensors under each foot: two under the toe area and two 
under the heel. These provide a good indication that both legs are in contact with the 
ground. The Bonten-Maru II's structure design and control system are used in experiments 
and evaluations of this research. 



5.2. Self-Localization: Defining Humanoid Robot's Orientation from Groping Result 

The end effector data obtained during groping process are calculated with the least-square 
method to result a linear equation as shown in Eq. (1). Here, distance and groping angle 
between the robot to the wall, described as L and (/), respectively, are defined by applying 
formulations shown at belows. At first, a straight line from the reference coordinates origin 
and perpendicular with Eq. (1), which described the shortest distance from robot to wall, is 
defined in Eq. (2), where the intersection coordinate in X-Y axes plane is shown in Eq. (3). 
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Groping angle (/) is an angle from X-axis of the robot reference coordinates to the 
perpendicular line of Eq. (2). Here, distance L and groping angle 0are shown in Eqs. (4) and 
(5), respectively (also refer Fig. 4). In this research, correction of the robot position and 
orientation are refers to values of L and </>. Eventually, correction of the robot's locomotion 
direction basically can be defined by rotating the robot's orientation to angle 90°- 0, so that 
robot's orientation becomes parallel with the wall's orientation. 



L = 



tan 



1 
a 



(4) 
(5) 



5.3 Correction of Humanoid Robot's Orientation and Locomotion Direction 

5.3.1 Correction of distance 

Figure 9 shows top view of structural dimensions of Bonten-Maru II and groping area of the 
robot's arm. This figure is used to explain formulations to define correction of diatance for 
groping front wall and right-side 

Groping front wall 

In groping front wall, position of the wall facing the robot creates possibility of collision 
during correction of the robot's orientation. Therefore, correction of robot's distance was 
simply performed by generating trajectory for legs to walk to backwards direction. Here, 
quantity of steps are required to define. The steps quantity are depends on distance of the 
robot to the wall, and calculations considering the arm's structural dimension and step size 
(length of one step) for the utilized humanoid robot's leg. The formulation to define 
quantity of steps is shown in following equation. 



n {L x <L<L n 
n-\ (L m <L<L t ) 



(6) 



Here, a is step quantity, and L is the measured distance (shortest distance) from the 
intersection point of right arm's shoulder joints to the wall, which obtained from groping 
result. Refer to Fig. 9, during process of searching for wall, only elbow joint is rotating while 
the two shoulder joints are remain in static condition. Here, Li is dimension from the 
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shoulder joints to the elbow joint, L t is the total length of arm from the shoulder joints to the 
end effector, and L3 is the step size of the robot's leg. Consequently, L m that indicate in Eq.6 
is defined from following equation: 



- + L, 



(?) 



In groping 
front wall >' 



A* 



I' 1 
lil 




m / 


h 


^ 1 1 


1- ^ 


1™ 1 




u^. 


? 



In groping \ 
right-side wall \ 



Left 



Right 



* ►! 



Fig. 9. Structural dimensions and groping area of the humanoid robot's arm. 

Groping right-side wall 

In groping right-side wall, correction of distance involves trajectory generation of legs to 
walk side-step away from the wall. However, if the groping angle <f> is O<0<45°, it is still 
possible for the robot to collide with the wall. In this case, the robot will walk one step to 
backward direction, before proceed to walk side-step. Eventually, if the groping angle <p is 
45°<0<9O O , the robot will continue to correct its position by walking side-step away from the 
wall. At this moment, the side-step size S is defined from Eq. (8). Here, L is the distance 
between the robot to the wall, while Lb is a parameter value which considered safety 
distance between the robot to the wall during walking locomotion. Parameter value of Lb is 
specified by the operator which depends on the utilized humanoid robots. 

S = (L b -L)sm</> (8) 

Continuously, from Eq. (8), boundary conditions are fixed as following Eqs. (9) and (10). Here, a 
and p are parameter values which consider maximum side-step size of the humanoid robot legs. 
Value of a is fixed at minimum side-step size, while ft is fixed at maximum side-step size. 



a {L b -L<0) 

(L b -Z)sin^ (L b -L>6) 

P {(L b - L) sin 4»p) 

[l b -Z)sin^ ((L b -L)^m(/)<P) 



(9) 



(10) 
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5.3.2 Correction of angle 

Correction of the robot's angles is performed by changing the robot orientation to 90 o -(j), so that 
the final robot's orientation is parallel with wall's surface orientation. Figure 10 (a) ~ (c) shows a 
sequential geometrical analysis of the robot's foot-bottom position during correction of angle. 
From this figure, X-Y axes is a reference coordinates before rotation, while X'-Y' axes is the new 
reference coordinate after the rotation. Here, a is distance from foot center position to the robot's 
body center position, while b is correction value to prevent flexure problem at the robot's legs. 
Position of the left foot bottom to correct robot's angle in X-Y axes plane are described as y/ and 5, 
as shown in Fig. 10 (b). In this research, value of y/ Is fixed to be half of the humanoid robot's step 
size, while value of 5 is defined from following equation. 

5 = 2a + b + y/ (11) 

Figures 11(a) and (b) are respectively shows geometrical analysis of the robot's position and 
orientation at X-Y axes plane before and after correction of distance and angle in groping 
front wall and groping right-side wall, based on groping result. Axes X-Y indicating 
orientation before correction, while axes X'-Y' are after correction is finished. 
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Fig. 10. Geometrical analysis of the robot's foot-bottom position during correction of angle. 
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Fig. 11. Geometrical analysis of humanoid robot's orientation in groping front wall and 
right-side wall. 
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6. Trajectory Generation in Groping Locomotion to Avoid Obstacle 

The formulation and optimization of joint trajectories for a humanoid robot's manipulator is quite 
different from standard robots' because of the complexity of its kinematics and dynamics. This 
section presents a formulation to solve kinematics problems to generate trajectory for a 21-DOF 
humanoid robot in the obstacle avoidance method. The detail kinematics formulations are applied 
within the algorithm of the groping-locomotion method. 

Robot kinematics deals with the analytical study of the geometry of a robot's motion with respect 
to a fixed reference coordinate system as a function of time without regarding the force/ moments 
that cause the motion. Commonly, trajectory generation for biped locomotion robots is defined by 
solving forward and inverse kinematics problems (Kajita et al., 2005). In a forward kinematics 
problem, where the joint variable is given, it is easy to determine the end effector's position and 
orientation. An inverse kinematics problem, however, in which each joint variable is determined 
by using end-effector position and orientation data, does not guarantee a closed-form solution. 
Traditionally three methods are used to solve an inverse kinematics problem: geometric, iterative, 
and algebraic (Koker, 2005). However, the more complex the manipulator's joint structure, the 
more complicated and time-consuming these methods become. In this paper, we propose and 
implement a simplified approach to solving inverse kinematics problems by classifying the robot's 
joints into several groups of joint coordinate frames at the robot's manipulator. To describe 
translation and rotational relationship between adjacent joint links, we employ a matrix method 
proposed by Denavit-Hartenberg (Denavit & Hartenberg, 1995), which systematically establishes 
a coordinate system for each link of an articulated chain (Hanafiah et al., 2005c). 



6.1 Kinematics analysis of a 3-DOF humanoid robot's arm 

The humanoid robot Bonten-Maru II has three DOFs on each arm: two DOFs (pitch and roll) at the 
shoulder joint and one DOF (roll) at the elbow joint. Figure 12 shows the arm structure and 
distribution of joints and links. This figure also displays a model of the robot arm describing the 
distributions and orientation of each joint coordinates. The coordinate orientation follows the 
right-hand law, and a reference coordinate is fixed at the intersection point of two joints at the 
shoulder. To avoid confusion, only the X and Z axes appear in the figure. The arm's structure is 
divided into five sets of joint-coordinates frames as listed below: 

2Jo '■ Reference coordinate. Z$ : Elbow joint roll coordinate. 

2a '■ Shoulder joint pitch coordinate. 2m '■ End-effector coordinate. 

Xi '■ Shoulder joint roll coordinate. 

Consequently, corresponding link parameters of the arm can be defined as shown in Table 
2. From the Denavit-Hartenberg convention mentioned above, definitions of the 
homogeneous transform matrix of the link parameters can be described as follows: 



\T = Rot(z, 0)Trans(O,O / ^)Trans(fl / O / O)Rot(x, a) 



(12) 
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Table 2. Link parameters of the robot arm. 
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Here, variable factor Z - is the joint angle between the Xm and the X; axes measured 
about the Z ? axis; di is the distance from the Xm axis to the X; axis measured along the 
Zi axis; oc\ is the angle between the Z; axis to the Zi-i axis measured about the Xj_i axis, 
and Z z is the distance from the Z z axis to the Z{.\ axis measured along the Xm axis. 
Here, link length for the upper and lower arm is described as h and h, respectively. 
The following Eq. (13) is used to obtain the forward kinematics solution for the robot 



Jr** 





Fig. 12. Arm structure and configurations of joint coordinates at the robot arm of Bonten- 
Maru II. 
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(13) 



The end-effector's orientation with respect to the reference coordinate (%R ) is shown in 
Eq. (14), while the position of the end effector (°Ph) is shown in Eq. (15). The position of 
the end effector in regard to global axes P x , P y and P z can be define by Eq. (16). Here, s ? 
and Cj are respective abbreviations of sindj and cosdj, where (z=l,2,...,n) and n is equal to 
quantity of DOF. 
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(14) 
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hs 2 +l 2 s 23 
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(15) 
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P x a rm =^l( / l c 2 + ^23) 
P z a rm = " c l (^2 + h^23 ) 



(16) 



As understood from Eqs. (14) and (15), a forward kinematics equation can be used to 
compute the Cartesian coordinates of the robot arm when the joint angles are known. 
However, in real-time applications it is more practical to provide the end effector's position 
and orientation data to the robot's control system than to define each joint angle that 
involved complicated calculations. Therefore, inverse kinematics solutions are more 
favorable for generating the trajectory of the humanoid robot manipulator. To define joint 
angles ftarm, fearm, fearm in an inverse kinematics problem, at first each position element in 
Eq. (16) is multiplied and added to each other according to Eq. (17), which can also be 
arranged as Eq. (18). Thus, fearm is defined in Eq. (19). 

P Imn 2 + P v 2 + P z ^ 2 = h 2 + U 1 + 2/i/ 2 c 3 (17) 

x arm y arm z arm i z i z j \ / 

_ P *arm +P J arm + P zarm ~ ( 7 1 + 7 2 ) _ 
3 2/!/ 2 



^3 arm = Atan2 I ± Vl - C 2 , C I (19) 



Referring to the rotation direction of ftamv if smfearm is a positive value, it describes the inverse 
kinematics for the right arm, while if it is a negative value it described the left arm. Consequently, 
ftarm is used to define ftarm, as shown in Eqs. (20) ~ (22), where newly polar coordinates are defined 
in Eq. (22). Finally, by applying formulation in Eqs. (23) and (24), ft arm can be defined as in Eq. (25). 

k x = l x +/ 2 c 3 , k 2 = -l 2 s 3 (20) 



kiC 2 +k 2 S2, p y -k 2 c 2 —k x s 2 (21) 

<p = Atan2 (k u k 2 ) (22) 



Z^ ? Zi_. = Atan2 { Pxz ,p y ) (23) 



02arm = Atan2 
#2arm = Atan2 (p xz ,Py )" Atan2 (k x ,k 2 ) (24) 

^^,^-l = Atan2( Px , jPz ) (25) 

V Pxz Pxz J 

6.2 Kinematics analysis of a 6-DOF humanoid robot's leg 

Each of the legs has six DOFs: three DOFs (yaw, roll and pitch) at the hip joint, one DOF 
(pitch) at the knee joint and two DOFs (pitch and roll) at the ankle joint. In this research, we 
solve only inverse kinematics calculations for the robot leg. A reference coordinate is taken 
at the intersection point of the three-DOF hip joint. In solving calculations of inverse 
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kinematics for the leg, just as for arm, the joint coordinates are divided into eight separate 
coordinate frames as listed bellow. 

Zo '■ Reference coordinate. 

2a '■ Hip yaw coordinate. 

2ji '■ Hip roll coordinate. 

Zs '■ Hip pitch coordinate. 

2a '■ Knee pitch coordinate. 

2j5 '■ Ankle pitch coordinate. 

Xe '■ Ankle roll coordinate. 

2m '■ Foot bottom-center coordinate. 

Figure 13 shows the structure and distribution of joints and links in the robot's leg. This figure 

also shows a model of the robot leg that indicates the distributions and orientation of each set of 

joint coordinates. Here, link length for the thigh is k, while for the shin it is h. The same 

convention applies for the arm link parameter mentioned earlier. Link parameters for the leg are 

defined in Table 3. Referring to Fig. 13, the transformation matrix at the bottom of the foot ( 6 h T ) is 

an independent link parameter because the coordinate direction is changeable. Here, to simplify 
the calculations, the ankle joint is positioned so that the bottom of the foot settles on the floor 
surface. The leg's orientation is fixed from the reference coordinate so that the third row of the 
rotation matrix at the leg's end becomes like following: 

P z leg = [0 if (26) 

Furthermore, the leg's links are classified into three groups to short-cut the calculations, 
where each group of links is calculated separately as follows. 

i) From link to link 1 (Reference coordinate to coordinate joint number 1). 

ii) From link 1 to link 4 (Coordinate joint number 2 to coordinate joint number 4). 

iii) From link 4 to link 6 (Coordinate joint number 5 to coordinate at the foot bottom). 
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Fig. 13. Structure and configurations of joint coordinates at the robot leg of Bonten-Maru II. 
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Table 3. Link parameters of the leg. 

Basically, i) is to control leg rotation at the Z-axis, ii) is to define the leg position, while iii) is 
to decide the leg's end-point orientation. A coordinate transformation matrix can be 
arranged as below. 



%T = \T\T\T ={\T)C 2 T \T lT)C 5 T%T b h T) 



(27) 



Here, the coordinate transformation matrices for 4 T and h T can be defined as Eqs. (28) 
and (29), respectively. 



lT = \T\TlT : 
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iT = 4 5 T 5 6 T 6 h T = 



C 5 C 6 




-c 5 s 6 -s 5 

-s 5 s 6 c 5 
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(29) 



The coordinate transformation matrix for^r, which describes the leg's end-point position 
and orientation, can be shown with the following equation. 



(30) 
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From Eq. (26), the following conditions were satisfied. 



'23 



'31 



'32 







'33 



= 1 



(31) 



Hence, joint rotation angles 0ii eg ~06ieg can be defined by applying the above conditions. 
First, considering i), in order to provide rotation at the Z-axis, only the hip joint needs to rotate in the 
yaw direction, specifically by defining fti eg . As mentioned earlier, the bottom of the foot settles on 
the floor surface; therefore, the rotation matrix for the leg's end-point measured from the reference 
coordinate can be defined by the following Eq. (32). Here, ftu eg can be defined as below Eq. (33). 
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° h R = Rot(zA l eg) : 
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(33) 



Next, considering ii), from the obtained result of 0n eg/ ^r is defined in following equation: 



O rp _ 
h 1 ~ 



~ s \ 


~ c \ 


^ 


c \ 


~ s \ 


^le 








1 P 7 A» 











z leg 
1 



(34) 



Here, from constrain orientation of the leg's end point, the position vector of joint 5 is 
defined as follows in Eq. (35), and its relative connection with the matrix is defined in Eq. 
(36). Next, equation (37) is defined relatively. 
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(35) 
(36) 

(37) 



(38) 



To define joint angles feeg, feieg, #4ie g , Eq. (38) is used, and it is similar to the calculation for 
solving inverse kinematics using Eq. (16) for the arm. Therefore, the rotation angles are 
defined as the following equations. 



0, 



il 



7 4leg = Atan2 ±Vl-C z ,C 

# 3 , eg = Atan2 ( p xz leg , p y ^ ) + Atan2 (*, , k 2 ) 
2] = Atan2 {p x , , p z . ) 



(39) 

(40) 
(41) 



Eventually, C, j? xzle2 ,ki,k 2 are defined as follows. 
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2 , - 2 , - 2 /7 2 , 7 2x 

^leg + ^ leg + Pz leg ~(h +h ) 

2/1/2 

Pxzi eg =yJPxi eg 2 +Pzi eg 2 (43) 

111 24 (M) 

/C2 — — ^2^4 

Finally, considering iii), joint angles feeg and de leg are defined geometrically by the following 
equations. 

*5,e g =-03le g -04leg (45) 

*6l eg = -^2 leg (46) 

6.3 Interpolation of Manipulator's End-Effector 

A common way of making a robot's manipulator to move from start point Po to finish point P/in a 
smooth, controlled fashion is to have each joint to move as specified by a smooth function of time. 
Each joint starts and ends its motion at the same time, thus the robot's motion appears to be 
coordinated. To compute these motions, in the case that start position Po and end position P/are 
given, interpolation of time t using polynomial equations is performed to generate trajectory. In 
this research, we employ degree-5 polynomial equations as shown in Eq. (47) to solve 
interpolation from Po to P/. Time factors at Po and P/are expressed as to=0 and tf, respectively. 

P(t) = a + a x t + a 1 t 1 + a 3 t 3 + a 4 t 4 + a 5 t 5 (47) 

Here, coefficient a\ (i=0, 2,.., 5) is defined by solving deviations of 

P (0 ), P (0 ), P (0 ), P (t f ) P (t f ) P (t f ) . In this study, velocity and acceleration at P and P f 

are defined as zero; only the position factor is considered as a coefficient for performing 
interpolation. Finally the interpolation equation is defined by Eq. (48), where 
f current time 



t f motion time 



P(t) = P +\0{p f -P )u 3 - 15 (P f -P )u 4 +6(p f -P )i 5 (48) 

7. Experimental Results and Discussion 

7.1 Experimental Procedure 

Experiments were conducted in conjunction with the groping locomotion experiments. 
Initially, a series of motion programs were created and saved in the robot's control system. 
Before performing the experiments, a simulation using animation that applies GNUPLOT 
was performed for analysis and confirmation of the robot joint's trajectory generation. 
Figure 14 presents the animation screen of the robot's trajectory, which features a robot 
control process and motion instructor process. This figure also shows the path planning of 
humanoid robot navigation performed in the experimet. Each joint's rotation angles are 
saved and analyzed in a graph structure. This is to ensure the computation of joints rotation 
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angle was correct and according to result of groping locomotion. For example, graphs for 
the left and right leg are plotted in Fig. 15 and Fig. 16 respectively during obstacle 
avoidance. The graphs show the smooth trajectory of the rotation angles at each leg's joint. 
In this experiment, the wall is positioned at the robot's front and its right side, while an 
obstacle is on its left side. The obstacle height is about same with the robot shoulder. During 
experiments, at first the robot performing groping locomotion to define groping angle, then 
continuously performs the obstacle avoidance. The experiment is conducted in autonomous 
way and the performance is evaluated by observation. In order to recognize objects, six-axis 
force sensors were attached to the robot arms. The utilized force sensors are designed to 
detect three force components in each axial direction, with the other three components of 
moment around each axis operating simultaneously and continuously in real time with high 
accuracy. The maximum loads at the XY-axes are 400 N, while at the Z-axis it is 200 N. 
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Fig. 14. Animation of the robot's trajectory and path planning of the experiment 
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Fig. 15. Rotation angle of the left leg joints in the obstacle avoidance. 
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Fig. 16. Rotation angle of the right leg joints in the obstacle avoidance. 



7.2 Results of Humanoid Robot Locomotion 

Figure 17 shows sequential photographs of the actual robot's locomotion during experiments on 
groping front wall, Meanwhile Fig. 18 shows sequential photographs of groping right-side wall 
experiment. Consiquently, the humanoid robot performed the obstacle avoidance as shown in Fig. 
19. The experimental results reveal that the robot's arm and legs move in a smooth and controlled 
motion to perform tasks in groping locomotion and obstacle avoidance. The formulations from the 
proposed groping locomotion algorithm guided the robot locomotion to recognize wall's 
orientation and correct robot's distance and angle based on the groping result. Meanwhile 
formulations in obstacle avoidance algorithm combined with groping locomotion algorithm 
recognize the presence of obstacle and perform suitable trajectory to avoid the obstacle. The 
proposed kinematics and interpolation formulation generate smooth trajectory for the arms and 
legs during performing locomotion in groping locomotion and obstacle avoidance experiments. 




Fig. 17. Sequential photograph of groping front wall experiment. 




Fig. 18. Sequential photograph of groping right-side wall experiment. 
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Fig. 19. Sequential photograph of obstacle avoidance in groping locomotion experiment. 



8. Conclusion 

The development of autonomous navigation system for humanoid robot to solve the problem 
of " working coexistence" of humans and robots is an important issue. It is apparent that the 
common living and working environment to be shared by humanoid robots is presently 
adapted mainly to human, and it cannot be expected that this will be significantly changed to 
suit the needs of robots. Hence, the problem of human-humanoid robot interaction, and 
humanoid robot-surrounding environment interaction are become the research topics that are 
gaining more and more in importance. Furthermore, contact interaction-based navigation 
system is practically significant for humanoid robots to accurately structure and recognize 
their surrounding conditions (Ellery, 2005, Salter et al., 2006). 

Research on groping locomotion in humanoid robot's navigation system has led to the 
proposal of a basic contact interaction method for humanoid robots to recognize and 
respond to their surrounding conditions. This research proposed a new obstacle avoidance 
method which applied reliable algorithms in a humanoid robot control system in 
conjunction with the groping-locomotion algorithm. The proposed method is based on 
contact interaction whereby the robot arms directly touch and analyze an object, with the 
aim of accomplishing the objective of developing an interaction method for the humanoid 
robot and its surroundings. Performance of the proposed method was evaluated by 
experiments using prototype humanoid robot Bonten-Maru II which force sensors are 
attached to its arms as end-effector to detect and recognize objects. 

The experimental results indicated that the humanoid robot could recognize the 
existence of an obstacle and could avoid it by generating suitable leg trajectories. The 
proposed algorithm was effectively operated in conjunction with the groping 
locomotion algorithm to detect and avoid obstacle in the correction area, which 
improved the performance of the groping locomotion. Regarding the motion of the 
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humanoid robot's arms, the proposed algorithm provides a good relationship between 
groping locomotion and obstacle avoidance. It demonstrates intelligent detection of 
most objects around the robot, enabling the robot's control system to effectively 
identify the object position and perform necessary locomotion. 

In the experiments with humanoid robot, autonomous motions of the robot's manipulators 
are managed to demonstrate. These satisfy the objective of this research to develop an 
autonomous navigation system for bipedal humanoid robot to recognize and avoid 
obstacles in groping locomotion. Consiquently, the proposed groping locomotion method 
clearly demonstrated two important tasks to solve in the autonomous navigation for 
walking robots: self-localization and obstacle avoidance. 

The proposed idea should contribute to better understanding of interactions between a 
robot and its surroundings in humanoid robot's navigation. Furthermore, future refinement 
of the proposed idea in various aspects will result in better reliability of the groping 
locomotion mechanism, enabling any type of anthropomorphic robots fitted with it to 
operate effectively in the real environments. It is anticipated that using this novel humanoid 
robot's navigation system technology will bring forward the evolution of human and 
humanoid robots working together in real life. 

9. Future Development: Development of Object Handling 

As mentioned in previous section, an autonomous navigation in walking robots requires 
that three main tasks be solved: self-localization, obstacle avoidance, and object handling. In 
current research, we proposed a basic humanoid robot navigation system called the 
" groping locomotion" for a 21-DOF humanoid robot, which is capable of defining self- 
localization and obstacle avoidance. 

In future work, we going to focus on development of the object handling. Although current robot 
hands are equipped with force sensors to detect contact force, they do not make use of sensors 
capable of detecting an object's hardness and/ or softness, nor can they recognize the shape that 
they grip. For a robot hand to grip an object without causing damage to it, or otherwise 
damaging the sensor itself, it is important to employ sensors that can adjust the gripping power. 
Recently, with the aim to determining physical properties and events through contact during 
object handling, we are in progress of developing a novel optical three-axis tactile sensor capable 
of acquiring normal and shearing force (Ohka et al., 2006). A tactile sensor system is essential as a 
sensory device to support the robot control system (Lee & Nicholls, 1999, Kerpa et al., 2003). This 
tactile sensor is capable of sensing normal force, shearing force, and slippage, thus offering 
exciting possibilities for application in the field of robotics for determining object shape, texture, 
hardness, etc. The tactile sensor system developed in this research is combined with 3-DOF 
humanoid robot finger system where the tactile sensor in mounted on the fingertip. 
Future work will involve further development of the contact-based humanoid robot 
navigation system project, applying the integrated system comprising the optical 
three-axis tactile sensor and robot fingers in humanoid robot's control system for 
object handling purposes. 
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1. Introduction 

We consider a two-link biped, a three-link biped, and a five-link planar biped without feet (with 
"point feet"). Their ankles are not actuated. The control torques are applied to the bipeds in the 
inter-link joints only. Then this family of bipeds is under actuated when only one leg tip touches 
the ground. It is difficult to control the walking of this kind of bipeds because they are statically 
unstable in single support. For example the vertical posture of these bipeds in single support is an 
unstable equilibrium state, as an equilibrium state of inverted pendulum. The operations of 
stabilization for the biped vertical posture, of balancing around this equilibrium posture, using 
only the inter-link torques, are also difficult. These problems are interesting from the point of view 
of dynamical stabilization of walking for bipeds with motions in saggital plane or (and) in frontal 
plane. They are also interesting from the biomechanical point of view. In the chapter, the problem 
of stabilization of the vertical posture for each mentioned above biped is studied. For each biped, a 
control law to stabilize the vertical posture is designed. 

Among the mechanical systems, the under actuated systems, which have fewer controls 
than configuration variables, represent a great challenge for the control. An active field of 
research exists, due to the applications of under actuated systems such as aircrafts, satellites, 
spacecrafts, flexible robots, inverted pendulums, legged robots. The under actuated systems 
are characterized by the under-actuation degree, which is the difference between the 
numbers of configuration variables and controls. The under-actuation degree for all our 
studied bipeds equals one in single support. 

The control laws to stabilize the vertical posture are designed, using the biped linear models 
and their associated Jordan forms. The feedback is synthesized to suppress the unstable 
modes. The restrictions imposed to the torques are taken into account explicitly. Thus, 
feedback control laws with saturation are designed. It is important for an unstable system to 
maximize the basin of attraction. Using the Jordan form to design the control law, we can 
obtain a large basin of attraction of equilibrium state. 
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With the aim to achieve fast walking gaits, some papers have been devoted to the study of 
walking mechanisms as the compass and the biped with point feet (see for example, (Spong et ah, 
2000); (Cambrini et ah, 2001); (Canudas et ah, 2002); (Aoustin & Formal'sky, 2003); (Chevallereau et 
ah, 2003); (Westervelt et al., 2003)). The study and control of walking and running gaits of these 
robots is a very interesting and simultaneously difficult problem. The challenge of the control law 
is to "suppress" the instability of these statically unstable and under actuated objects and also to 
reduce the time of transient oscillations. In (Cambrini et ah, 2001), it is shown that it is possible to 
track in single support stable trajectories with internal stability by a suitable choice of outputs for a 
two-link robot and for a five-link robot. The authors in (Canudas et ah, 2002); (Aoustin & 
Formal'sky, 2003); (Chevallereau et ah, 2003); (Chevallereau et ah, 2004) realize orbital stabilization 
for a five-link biped, also in the single support phase. For the family of bipeds with internal 
torques, it is possible dynamically to stabilize their specific walking gaits. For example, in (Grizzle 
et ah, 2001) authors, using the Poincare map, proved that it is possible to ensure the dynamical 
stability of a three-link biped under a control law being finite time convergent. In (Aoustin & 
Formal'sky, 2003), the convergence to a nominal cyclic motion is improved, by changing the step 
length or the trunk orientation. 

Usually the limits imposed on the torques are not taken into account explicitly. For the 
problem of posture stabilization we propose a strategy of control with restricted torques. 
The control law is defined such that the torques adjust only the unstable modes of the biped. 
The numerical investigations of nonlinear models of the mentioned bipeds with the 
designed control laws are presented. The efficiency of the designed controls is shown. 
In our opinion, the described approach here is useful for unstable systems of different kind. 
It is possible to apply this approach for the stabilization of inverted pendulums, for 
stabilization of monocycle (Beznos et ah, 2003); (Aoustin et ah, 2005); (Aoustin et ah, 2006); 
(Formal'sky, 2006); (Martynenko & Formal' sky, 2005). 

The organization of this chapter is the following. Section 2 is devoted to the model of the 
biped. It contains also the data of the physical parameters of the five-link biped. The linear 
model of the biped motion around the vertical posture is presented in Section 3. The 
statement of the problem is defined in Section 4. The control law for the two-link biped is 
designed in Section 5. The control laws for the three-link and five-link bipeds are developed 
in Sections 6 and 7 respectively. Section 8 presents our conclusion and perspectives. 

2. Model Description of the Planar Biped 

2.1 The dynamic model 

We consider an under actuated planar biped with n degrees of freedom and n - 1 actuators. Thus, 
the under-actuation degree for our biped equals one in single support. The generalized forces 
(torques) are only due to the actuators in the inter-link joints. The dynamic model of the biped 
single support motion is given by the following Lagrange matrix equation: 

D(q)q + C(q,q) + Fq + G(q) = Br (1) 

Here, q is the n x 1 configuration vector. Its coordinates are the absolute angle between 

the trunk and the vertical axis, and the n-1 actuated inter-link angles. D(q) is the nxn 
inertia positive definite matrix, C(q,q) is thenxl column of Coriolis and centrifugal 

forces. The matrix D(q) depends on the n-1 inter-link angles only. We assume that at 
each actuated joint there is a viscous friction. Let the friction coefficient f be identical in 
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all actuated joints, F = 



^ (n-l)xl 



"lx(n-l) 



where I n _i is a (n-l)x(n-l) unit matrix. G(q) 



is the nxl vector of the torques due to gravity. B is a constant nx(n-l) matrix, T is 
the (n-l)xl vector of the torques, applied in the knee and hip joints. The diagrams of 
the two-link biped (n = 2), the three-link biped (n = 3), and the five-link biped (n = 5) 

are presented in Figure 1. The model (1) is computed considering that the contact 
between the tip of the stance leg and the ground is an undriven pivot. But in reality there 
is a unilateral constraint between the ground and the stance leg tip: the ground cannot 
prevent the stance leg from taking off. We assume there is no take off and no sliding. 
Thus, it is necessary to check the ground reaction in the stance leg tip. Its vertical 
component R y must be directed upwards. We introduce the following equations 
applying Newton's second law for the center of mass of the biped to determine the 
ground reaction R ( R x , R ), 

R x =Mx c 
R y =My c +Mg 

Here, M is the total mass of the biped, x c and y c are the coordinates of the mass center of the 
biped. To check if the ground reaction is located in the friction cone, we have to calculate the 
ratio R x /Ry. 



Link I 



(2) 




Fig. 1. The three studied bipeds (diagrams). 



2.2 The physical parameters of dynamic model 

For the numerical experiments we use the physical parameters of the five-link biped 

prototype "Rabbit" (Chevallereau et ah, 2003). 

We assume that both legs are identical (see Figure 1, two last diagrams). The masses of the 

shins are: m { = m 5 = 3.2kg ; the masses of the thighs are: m 2 = m 4 = 6.8 kg ; the mass of the 

trunk is: m 3 =16.5kg. The lengths of the shins and the thighs are identical: 

1-l = 1 2 = 1 3 = 1 4 = 1 5 = 0.4 m ; the length of the trunk is: 1 3 = 0.625 m . The height of the biped 

equals 1.425 m , the total mass M equals 36.5 kg . 

The distances between the mass center of each link and the corresponding joint are the 

following: ^ =s 5 =0.127 m, s 2 =s 4 =0.163 m and s 3 = 0.2 m . 

The inertia moments around the mass center of each link are: ^=15= 0.0484 kg • m 2 , 
I 2 = I 4 = 0.0693 kg • m 2 and I 3 = 1.9412 kg • m 2 . The inertia of the rotor for each DC motor is 



30 Mobile Robots, Towards New Applications 

I = 3.32 -HT 4 kg -m 2 . 

All the gear ratios are identical and equal 50. The maximum value U of the torques equals 

150N.m. 

Using these values we have also calculated the corresponding values for the two-link and 
three-link bipeds. In Section 5, we calculate the parameters of the two-link biped. For 
example, the mass of the first link of the two-link biped (see Figure 1, first diagram) equals 
u a = m 3 + m 4 + m 5 . The mass of its second link (of the leg) equals u 2 = m 2 + m 1 + m 4 + m 5 . 

The length of the first link equals 1 3 + 1 4 + 1 5 , the length of its second link (of the leg) equals 
1 = 1 2 + l a = 1 4 + 1 5 . The distance r 1 between the unique inter-link joint and the center of mass 
of the first link equals 

r _ -m 3 s 3 +m 4 s 4 +m 5 (l 4 +s 5 ) (3) 

m 3 +m 4 +m 5 
The distance r 2 between the inter-link joint and the center of mass of the second link equals 
_ m 2 s 2 + m a (1 2 + gi ) _ m 4 s 4 + m 5 (1 4 + s 5 ) 

2 V*) 

m 2 +m 1 m 4 +m 5 

It follows from (3) and (4) that 

1-^X), l-r 2 >0, r 2 - ri >0 (5) 

In Section 6, we calculate the parameters of the three-link biped. The mass of each leg (see 
Figure 1, second and third diagrams) equals u 3 = m 2 + m^ = m 4 + m 5 . The mass of its trunk is 
m 3 . The length of each leg equals 1 = 1 2 H- 1 2 =1 4 +1 5 , the length of the trunk is 1 3 . The 
distance between the inter-link joint and the center of mass of each leg equals r 2 and is 
defined by the formula (4). The distance between the inter-link joint and the mass center of 
the trunk equals s 3 . 

3. Linear Model of the Planar Biped 

In this section, we present the matrix equation (1) linearized around the vertical posture of 
the biped, the state form of this linear model and its Jordan form. This Jordan form will be 
useful to define the control laws in the next sections. 
Let q e denote the configuration vector of the biped in the vertical posture. This vertical 

posture is an equilibrium position. This equilibrium point is q e = (0,7t) for the two-link 

biped, q e = (0,7t,7t) for the three-link biped, and q e = (0,7t,7t,7t,7t) for the five-link biped. 
The linear model is defined by the variation vector v = q - q e of the configuration vector q 
around the vertical equilibrium posture q e , 

D /V + Fv+G z v = Br (6) 

Here, D z is the inertia matrix for the configuration vector q e : D, = D(q e ) . G z is the Jacobian 
of the matrix G(q) computed at the equilibrium point q e . We will consider the following 
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constraint imposed on each torque: 

IrJ^U, (i=l, ...,n-l), U=const 

We deduce from (6) the state model with x = (v, v) : 

x = Ax + br 

Here, 



A = 



0. 



L 



-D^G, -DfFj 



b = 



u nx(n-l) 

J 



V D/B 



(7) 



(8) 



(9) 



Introducing a nondegenerate linear transformation x = Sy , with a constant matrix S , we 
are able to obtain the well-known Jordan form (Ogata, 1990) of the matrix equation (8): 

(10) 



y = Ay + dr 



where 



L = S a AS: 



, d = S~ 1 b 



(11) 



Here, X 1 ,...,X 2n are the eigenvalues of the matrix A, d is 2n x (n-1) matrix. Let us consider 
that the positive eigenvalues have the smaller subscript. For the two-link biped we will 
obtain X 1 > , Re^ < (i = 2, 3, 4) , for the three-link biped X 1 > , X 2 > , Re X. < 

(i = 3 - 6) , and for the five-link biped X. > (i = 1, 2, 3) , Re^ < (j = 4 - 10) . 



4. Problem Statement 

Let x = (here is a 2nxl zero-column) be the desired equilibrium state of the system (8). 
Let us design the feedback control to stabilize this equilibrium state x = under the 
constraint (7). In other words, we want to design an admissible (satisfying the inequality (7)) 
feedback control to ensure the asymptotic stability of the desired state x = . 
Let W be the set of the vector-functions T(t) such that their components T^t) 

(i = l, ..., n-1) are piecewise continuous functions of time, satisfying the inequalities (7). 
Let Q be the set of the initial states x(0) from which the origin x = can be reached, using 
an admissible control vector-function. Thus, the system (8) can reach the origin x = with 
the control T(t) e W , only when starting from the initial states x(0) e Q . The set Q is called 
controllability domain. If the matrix A has eigenvalues with positive real parts and the 
control torques T { (t) (i = l, ..., n-1) are limited, then the controllability domain Q is an 
open subset of the phase space X for the system (8) (see (Formal 7 sky, 1974)). 
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For any admissible feedback control r = T(x) (Ir^xJpU, i=l, ..., n-1) the corresponding 
basin of attraction V belongs to the controllability domain: VcQ. Here as usual, the basin 
of attraction is the set of initial states x(0) from which the system (8), with the feedback 
control T = T(x) asymptotically tends to the origin point x = as t — > °° . Some eigenvalues 

of the matrix A of the system (8), (10) are located in the left-half complex plane. The other 
eigenvalues of matrix A are in the right-half complex plane. We will design a control law, 
which " transfers' 7 these last eigenvalues to the left-half complex plane. 

The structure and the properties of this control law depend on the studied biped and its 
number of D.O.F. We will detail now these different cases. 

5. Two-Link Biped (n = 2) 

Here, we design a control law for a unique inter-link torque to stabilize the two-link planar 
biped, with a basin of attraction as large as possible. 

5.1 Control design for the two-link biped 

If the coefficient of the viscous friction f in the unique inter-link joint of the two-link biped is 
equal to zero, then the characteristic equation of the system (8), (10) with T = is 
biquadratic one 

a ^ 4 + ai X 2 +a 2 =0 (12) 

with 

a = detD z , a 2 = detG, = -|i 2 g 2 r 2 [^(1 -rj + jul 2 1] . 

The leading coefficient a of the equation (12) is positive because it is the determinant of the 
positive definite matrix D z . The free term a 2 of the equation (12) is negative because the 
difference 1 - r 2 is positive (see inequalities (5)). Therefore, the spectrum of the matrix A is 
symmetric with respect to the imaginary axis and it is naturally because with f = the 
system (8) is conservative. This property is true in the general case, i.e., for a biped with n 
links. In the case n = 2 (under condition f = 0), the matrix A has two real eigenvalues 
(positive and negative), and two pure imaginary conjugate eigenvalues. If f # , then the 
matrix A for n = 2 has one real positive eigenvalue, and three eigenvalues in the left-half 
complex plane. Let X 1 be the real positive eigenvalue, and let us consider the first scalar 

differential equation of the system (10) corresponding to this eigenvalue X 1 , 

y^y.+d.r (13) 

The determinant of the controllability matrix (Kalman, 1969) for the linear model (8) is not 
null, if and only if: 

r a (1 - r a ) [ J 2 + |i 2 r 2 (r 2 - r a )] + \i 2 (r 2 J x + 1J 2 ) * (14) 

Here ] 1 and J 2 are the inertia moments of the first and second links respectively around their 
mass centers. According to inequalities (5), the differences 1 - r 1 and r 2 - r 1 are positive. 
Therefore, the inequality (14) is satisfied and then (8) is a Kalman controllable system, then the 
scalar inequality d a ^ is verified. The controllability region Q of the equation (13) and 
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consequently of the system (10) is described by the following inequality (Formal 7 sky, 1974), 

|yi|<l d # as) 

Thus, the controllability domain (15) is a strip in the space Y or X. 

We can "suppress" the instability of coordinate yi by a linear feedback control, 

r = 7 y 1 (16) 

with the condition, 

A, 1 +d 1 y 1 <0 (17) 

For the system (8), (9) with n=2 under the feedback control (16) only one eigenvalue A 1 
(positive one) of the matrix A is replaced by a negative value A, a + d 1 y 1 . The eigenvalues X 2 , 
X 3 , X 4 do not change. 

If we take into account the constraints (7), we obtain from (16) a linear feedback control with 
saturation, 

U, if yy, > U 

r=r(yj= Wl/ if | Wl |<u as) 

-U, if 7y a <-U 

It is possible to see that if ly^ld-JU/^ (see inequality (15)), then under condition (17) the 
right part of the equation (13) with the control (18) is negative when y 1 > and positive 
when y 1 < . Consequently, if |y a (0)| < |d 1 |U/?i 1 , then the solution y a (t) of system (13), (18) 
tends to as t->«>. But if y 1 (t)^0, then according to the expression (18) r(t) — > as 
t -> oo . Therefore, the solutions yi(t) (i = 2, 3, 4,) of the second, third and fourth equations of 
the system (10) with any initial conditions yi(0) (i = 2, 3, 4,) converge to zero as t — > °° 
because ReX { < for i = 2, 3, 4 . Thus, under the control (18) and with the inequality (17), 

the basin of attraction V coincides with the controllability domain Q, (Formal 7 sky, 1974); 
(Grishin et ah, 2002): V = Q. So, the basin of attraction V for the system (8), (18) is as large as 
possible and it is described by the inequality (15). 
Note that the variable y l depends on the original variables from vector x according to the 

transformation x = Sy or y = S" 1 x. Due to this, formula (18) defines the control feedback, 
which depends on the vector x of the original variables. 

According to Lyapounov's theorem, see (Slotine & Li, 1991), the equilibrium q = q e of the 
nonlinear system (1) is asymptotically stable under the control (18) with some basin of attraction. 

5.2 Numerical results for the two-link biped 

We use the parameters defined in Section 2, the expressions (3), (4) and the friction 
coefficient f = 6.0 N • m/s to compute the parameters of the dynamic model for the two-link 

biped. The eigenvalues of the matrix A for the used parameters are the following: 

X a = 3.378 , A 2 = -3.406 , X 3A = -1.510 ± 3.584i . 
In simulation the control law (18) is applied to the nonlinear model (1) for the two-link biped. For 
the initial configuration q(0) = [ 1.94°; 180°] the graphs of the variables q^t) and q 2 (t) as functions 
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of time are shown in Figure 2. At the end the biped is steered to the vertical posture. All the 
potential of the actuator is applied at the initial time, as shown in Figure 3. The basin of attraction 
for the nonlinear system depends on the feedback gain y, which is chosen as -10000 in our 
numerical experiment. For the linear case, the relation (15) gives the maximum possible deviation 
of the two-link biped from the vertical axis 7.32° . But for the nonlinear model the maximum 
possible deviation is close to 1.94° . For each numerical experiment, we check the ground reaction 
in the stance leg tip to be sure that its vertical component is directed upwards. Figure 4 shows that 
during the stabilization process of the two-link biped, the vertical component of the ground 
reaction is always positive. Initially this component of the ground reaction is / Targe ,/ and equals 
the weight of the biped at the end of the process. 

We tested our strategy for different values of the actuator power. In fact, we considered 
different values U for the maximum torque T . Figure 5 shows that the allowable deviation 
of the biped from the vertical posture increases nonlinearly with increasing of the torque 
maximum U. And this allowable deviation is limited. For the linear model this dependence 
is linear function of the torque maximum U (see inequality (15)). 
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Fig. 2. Stabilization of the two-link biped in vertical posture, q a (t) — > and q 2 (t) ■ 
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Fig. 3. Stabilization of the two-link biped in vertical posture, torque in the inter-link joint. 
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Fig. 4. Stabilization of the two-link biped in vertical posture, vertical component of the 
ground reaction in the stance leg tip. 
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Fig. 5. Stabilization of the two-link biped, maximum allowable deviation of the biped from 
the vertical posture as a function of the maximum amplitude of the actuator torque. 

6. Three-Link Biped (n = 3) 

In this paragraph, we design a control law for the two inter-link torques to stabilize the 
vertical posture of the three-link planar biped in single support. 

6.1 Control design for the three-link biped 

If the coefficient of the viscous friction f = , the characteristic equation of the matrix A of 
the linear model (8), (9) of the biped is bicubic one 

a p 3 +a 1 p 2 +a 2 p + a 3 =0, p = X 2 (19) 
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with 

a =detD / , a 3 =detG l =[i 2 m 3 g\s 3 [[i 2 (\-Y 2 ) + \(^ 2 +m 3 )] 

The system (8), (9) describes small oscillations of the biped. According to Sylvester's 
theorem (Chetaev, 1962), (Chetaev, 1989) the roots p 1 , p 2 , p 3 of its characteristic equation 
(19) are real values. Thus, the spectrum of the matrix A with f = is symmetric with respect 
to the imaginary axis. The leading coefficient ao of the equation (19) is positive because it is 
the determinant of the positive definite matrix D z . The free term a 3 of the equation (19) is 
positive because the difference 1 - r 2 is positive (see the inequalities (5)). Thus, according to 

Viet formula p!P 2 p 3 = — - / the product p!p 2 p 3 is negative. This means that one or three 

a 

roots are negative. In the first case, two roots are positive, in the second case, there are no 
positive roots. But at least one root for the studied unstable system must be positive. 
Consequently, two roots, for example p a and p 2 , are positive and one is negative. Then 
spectrum of the matrix A contains two real positive eigenvalues, two real negative 
eigenvalues and two pure imaginary conjugate eigenvalues. If f ^ , then spectrum of the 
matrix A contains two values in the right-half complex plane, and four values in the left-half 
complex plane. 

Let X { , (i = 1, 2) be real positive eigenvalues of the matrix A (with f ^ ), and let us 
consider the first two lines of the matrix equation (10) corresponding to these eigenvalues, 

Yi = KYi + d n r i + d i2 r 2 (20) 

y 2 = ^ 2 y 2 +d 21 r 1 +d 22 r 2 

The torques are chosen such that, the instability of both coordinates y { , (i = 1, 2) is 
suppressed by choosing the feedback as, 

d 11 r 1 +d 12 r 2 =Yiyi pi) 

d 21 r 1 +d 22 r 2 = y 2 y 2 

with both conditions, 

A,+Yi«> (i=l,2) (22) 

Calculating both torques T a and T 2 from the algebraic equations (21), we obtain 



_ Y 1 d 22 y 1 -y 2 d 12 y 2 _ 



d n d 22 -d 12 d 21 (23) 

a 11 a 22 a 12 a 21 

We assume here that the denominator in the expressions (23) is not zero. 
For the system (10) with n=3 under the feedback control (23) two positive eigenvalues X 1 
and X 2 of the matrix A are replaced by negative values X 1 + y 1 and X 2 + y 2 respectively. All 
another eigenvalues do not change. 
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Taking into account the constrains (7), the applied torques are, 

U, if r°( yi ,y 2 )>u 



^=Ti(YuV2) = 



r?(y„y 2 ), if |r°( yi ,y 2 )|<u (i = i, 2) 

-U, if r°( yi ,y 2 )<-U 



(24) 



Under the control law (24), the equilibrium point x = is asymptotically stable for the 
system (8), (9) in some basin of attraction VcQ. 

Using Lyapounov's theorem (Slotine, 1991), we can prove that the equilibrium point q = q e 
of the nonlinear system (1), (24) is asymptotically stable as well. 

6.2 Numerical results for the three-link biped 

We use the parameters defined in Section 2 and the friction coefficient f = 6.0Nm/s to 

compute the parameters of the dynamic model for the three-link biped. The eigenvalues of 
the matrix A for the used parameters are the following 

X 1 = 3.506 , K = 1-343 , X 3 = -3.326 , A, 4 = -5.231 , X 5>6 = -1.133 ± 3.548i 
In simulation with the control law (24) applied to the nonlinear model (1) of the three- 
link biped it is possible to start with an initial configuration q(0) = [2.9°; 180°; 180°] . 

Thus, adding a joint in the haunch between the trunk and the stance leg leads to 
increasing the basin of attraction. The graphs of the variables q a (t), q 2 (t) and q 3 (t) as 

function of time are shown in Figure 6. At the end, the biped is steered to the vertical 
posture. The maximum torque of the actuator is applied at initial time, as shown in 
Figure 7. Figure 8 shows that during the stabilization process of the three-link biped 
the vertical component of the ground reaction is always positive and equals the weight 
of the biped at the end of the process. 
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Fig. 6. Stabilization of the three-link biped in vertical posture, q 1 (t)^0, q 2 (t)^-7i and 
q 3 (t) —>-n as t ^ oo . 
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Fig. 7. Stabilization of the three-link biped in vertical posture, torques in the inter-link joints. 
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Fig. 8. Stabilization of the three-link biped in vertical posture, vertical component of the 
ground reaction in the stance leg tip. 



7. Five-Link Biped (n = 5) 

In this section, we design a control law for four inter-link torques to stabilize the vertical 
posture of the five-link planar biped without feet in single support. 
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7.1 Control design for the five-link biped 

If the coefficient of the viscous friction f = , the characteristic equation of the matrix A is 
the following 

a p 5 +a 1 p 4 +a 2 p 3 +a 3 p 2 +a 4 p + a 5 =0, p = X 2 (25) 

with 

a = detD, , a 5 = detG, = 

-g 5 m 1 m 3 s 1 s 3 (m 1 l 2 + m 2 s 2 ) [(2m 2 + m 3 ) l x + m 1 (21 a - s 1 )] [(ir^ + m 3 ) 1 2 + m 2 (21 2 - s 2 )] 

According to Sylvester's theorem (Chetaev, 1962), (Chetaev, 1989) the roots p 1 , p 2 , p 3 , 

p 4 , p 5 the characteristic equation (25) are real values. The leading coefficient a of the 

equation (25) is positive. The free term as is negative because the differences 21 1 -s a and 

2L.-S2 are positive. Thus, according to Viet formula P1P2P3P4P5 = — -/ the product 

a 

P1P2P3P4P5 i s positive. This means that two or four roots are negative and respectively 
three roots or one root are positive. It is possible to prove (see (Chetaev, 1962), (Chetaev, 
1989)) that the three roots, for example p 1 , p 2 , p 3 are positive and two roots are 
negative. Then the spectrum of the matrix A contains three real positive eigenvalues, three 
real negative eigenvalues and two pairs of pure imaginary conjugate eigenvalues. If f ^ , 
then spectrum of the matrix A contains three values in the right-half complex plane, and 
seven values in the left-half complex plane. 

Let X { , (i = l, 2, 3) be real positive eigenvalues. Similarly to the previous cases let us 
consider the first three lines of the system (10) corresponding to these three positive 
eigenvalues, 

Yi = Kyi + d n r i + d 12 r 2 + d 13 r 3 + d 14 r 4 

y 2 = X 2 y 2 + d^r, + d 22 r 2 + d 23 r 3 + d 24 r 4 ( 26 ) 

73 = KYs + d 31 F l + d 32 r 2 + d 33 F 3 + d 3 4 r 4 

Now we want to suppress the instability of the three variables y. , (i = 1, 2, 3) . It can be 
achieved by choosing the controls T { (i = 1, 2, 3, 4) such that 



d n r\ +d 12 r 2 +d 13 r 3 +d 14 r 4 = y lYl 
d 21 r 1 +d 22 r 2 +d 23 r 3 +d 24 r 4 = y 2 y 2 

d 3i r i +d 32 r 2 + d 33 r 3 + d 3 4 r 4 = YsYs 



(27) 

d 2i r i +d 22 r 2 +d 23 r 3 +d 24 r 4 = y 2 y 2 



with the three conditions, 

^i+Ti<0 (i=l/2,3) (28) 

For the system (8), (9) with n=5 under the control, defined by equalities (27) and 
inequalities (28), three positive eigenvalues \ , X 2 and X 3 of the matrix A are replaced by 

negative values X 1 + y 1 , X 2 + y 2 and X 3 + y 3 respectively. All another eigenvalues do not 
change. 
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The system (27) contains three algebraic equations with four unknown variables T lf T 2 , T 3 
and T 4 . Therefore, the system (27) has an infinite number of solutions. We find a unique 
solution of the system (27) at each step of the control process by minimizing the following 
functional 

J = max In I ( 29 ) 

i=l,...,4 ' ' 

This yields the torques if (y 1 , y 2 , y 3 ) (i = l, 2, 3, 4). Finally, we apply the torques 
TiiYi' J i' J-i) > (i = 1/ 2, 3, 4) , limited by the same value U, 



r i= r i(yi/y 2 /y3) : 



U, if r°( yi ,y 2 ,y 3 )>U 

i/y 2 /y 3 )/ if | r i(yi/y 2 /y 3 )|^ u 

-U, if r°( yi ,y 2 ,y 3 )<-U 



r°( yi ,y 2 ,y 3 ), if |rT( yi ,y 2 ,y 3 )|<u (i = i, 2, 3, 4) (30) 



The torques in (30) ensure asymptotical stability of the equilibrium y. = (i = 1, 2, 3) of the 

first, second and third equations of the system (10) (i.e., of the system (26)), if the initial state 
belongs to some basin of attraction V in the three-dimensional space y 1 , y 2 , y 3 . However, 

the equilibrium point y. = (i = 4, ..., 10) of the fourth - tenth equations of the system (10) 

is also asymptotically stable for all initial conditions y t (0) (i = 4, ..., 10) because ReX { < 

for i = 4,...,10. Note that Vi ^0 (i = l, 2, 3) as t->°°, if (y a (0), y 2 (0), y(0)) c V . 

Consequently, according to the expressions (27), (29), (30) r^t) — » (i = 1, 2, 3, 4) as t — > °o . 

Thus, under the control (30) and with the conditions (28), the origin x = is an 
asymptotically stable equilibrium of the system (8) with some basin of attraction VcQ. 
The variables y { (i = 1, 2, 3) depend on the original variables from vector x according to 
the transformation y = S _1 x . Due to this, the formula (30) defines the control feedback, as a 
function of the vector x of the original variables. 

7.2 Numerical results for the five-link biped 

The eigenvalues of the matrix A for the used parameters and f = 6.0 N • m/s are the 

following 

^=5.290, X 2 =3.409, X 3 = 1.822, X 4 = -5.824, ^ 5 =-6.355, X 6 =-11.342/ 

X 7/8 = -1.162 + 3.512i, X 9 =-0.610, X 10 = -3.303- 

In simulation with the control law (30) applied to the nonlinear model (1) of the five-link biped, it 

is possible to start with an initial configuration q(0) = [1.7°; 180°; 180°; 180°; 180°] . The angles 

q a (t) and q 2 (t) , q 3 (t) , q 4 (t) , q 5 (t) as functions of time are shown in Figures 9, 10, 11. At the 

end the biped is steered to the vertical posture. All the potential of the actuator is applied at initial 
time, as shown in Figures 12, 13. 

Figure 14 shows that during the stabilization process of the five-link biped, the vertical 
component of the ground reaction is always positive and equals the weight of the biped at 
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the end of the process. 




Fig. 9. Stabilization of the five-link biped in vertical posture, q a (t) — > as t — > < 
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Fig. 10. Stabilization of the five-link biped in vertical posture, q 2 (t)^-7t, q 3 (t)— >-rc as 

t->oo. 




Fig. 11. Stabilization of the five-link biped in vertical posture, q 4 —> -n , q 5 -> -n as t — > oo . 
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Fig. 12. Stabilization of the five-link biped in vertical posture, torques in the inter-link joints 
1 and 2. 
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Fig. 13. Stabilization of the five-link biped in vertical posture, torques in the inter-link joints 
3 and 4. 
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Fig. 14. Stabilization of the five-link biped in vertical posture, vertical component of the 
ground reaction in the stance leg tip. 



Biped Without Feet in Single Support: Stabilization of the Vertical Posture with Internal Torques 43 

8. Conclusion 

We defined here control strategies to stabilize the equilibrium vertical posture of a two-link, 
a three-link and a five-link biped, explicitly taking into account the limits imposed on the 
amplitudes of the control torques. For each biped we use the Jordan form of its linear model 
to extract the unstable modes that we want to suppress with the feedback control. For the 
two-link biped, the control law is optimal. This means that the basin of attraction for its 
linear system is as large as possible, i.e., it coincides with the controllability domain. For the 
five-link biped several choices of torques are allowable because we have four torques and 
three unstable modes. Therefore, we define a criteria to compute these torques. All the 
numerical results in this paper are realistic. A perspective for the case of the three-link biped 
is to define a control law, for which the basin of attraction is as large as allowable. 
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1. Introduction 

Recently, humanoid research and development are widely under way. There are, however, still a 
lot of problems we have to solve. One fundamental problem is contact with a human. Robots 
coexisting with human beings have contact with human on a daily basis, and they are required to 
be entirely safe. Another fundamental problem for human coexisting robots is the diversity of 
humans' fields; diversity means that of tasks and that of the environment. We propose human- 
like body structure as a possible solution. It is comparatively easy to install physical flexibility to 
musculoskeletal robots, because their joints are passive. Muscle driven structure has advantage 
for increasing number of serially connected joints based on the 'coupled-drive 7 mechanism 
(Hirose et al., 1989). This paper presents the concept of our new humanoid Kotaro, describes the 
mechanical design including the actuation system and sensors, shows the demonstrations 
performed at the EXPO' '05, and discusses the future perspectives of the research. 

2. Humanoid and Human in Year 2020 

The title of the chapter goes " aiming at the future in 15 years 7 time/' This comes from the 
theme of " Prototype Robot Exhibition" at the World EXPO' 05. The exhibition was a part of 
" Project for the Practical Application of Next-Generation Robots" organized by NEDO 
(http://www.nedo.go.jp/english/). The purpose of the exhibition was 'to look for original 
technologies from Japan that could lead to the creation of robots with application in a wide 
variety of different environments, including households and offices, by 2020/ according to 
NEDO. Our proposal for the aim is musculoskeletal humanoids. Humanoids need to be 
softer and safer. They will also have to adapt to various environments and tasks. 
Musculoskeletal humanoids with many sensors have potential to solve the problems. In this 
section, the characteristics of our proposal are described. 

2.1 Interaction with Humans 

(a) Physical softness 

A robot in a human environment needs physical softness to avoid hurting humans and 
surroundings, as well as force-feedback control such as impedance control (Hogan, 1985). A 
human can control mechanical impedance by antagonistic muscles (Hogan, 1984). We could 
consider the softness of structure. Human's bone, which is not as stiff as metals, gives us an 
inspiration. A basis of controlling machines has been based on precision and rigidity. Assuming 
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mechanical flexibility may need a new paradigm of design and control of machines; human and 
animals do not seem to use the same basis in design and control of their body. 

(b) Sensing of contact information 

Tactile sensors are necessary for human environment. Mechanical softness of the surface is also 
important for contact with human. A problem related to tactile sensors of humanoid robots is 
difficulty of covering whole-body; covering around joints, fitting to complex surfaces, and 
difficulty of wiring. In this paper, we propose two types of tactile sensors. One is fleshy soft 
sensor and the other is bandage type sensor (see 5). 

(c) Multimodal communication 

There will be also social interaction with human. Robots ought to have visual and auditory senses 
and vocal function. We also have to think about the design of human-coexisting robots. 

2.2 Diversity of Human's Field 

There are numerous variety of tasks people expect robots to do at their houses. The versatility 
will greatly help the robots become widely used. The robots will need to adapt to unarranged 
environments. The necessity and usefulness of human-form (of the robots) is this. The number of 
joints is very important for achieving versatility. A human unconsciously uses a large number of 
his joints efficiently. The range of degrees of freedom (DOF) of current humanoid robots is from 
about 20 to about 40, while a human has more than 200 DOF. 



2.3 Musculoskeletal Humanoid Approach 

We have designed and developed a new musculoskeletal humanoid Kotaro (Fig.l), to show 
the possibility of musculoskeletal humanoid at the exhibition at the EXPO (Mizuuchi et al., 
2005b). Kotaro has a passive skeletal structure with many joints and motor-actuated muscles, 
and it has numerous sensors including muscle-tension sensors, rotary encoders as mus-cle- 
length sensors, motor-current sensors, tactile sensors, vision, audition, gyros, accelerometers, 
and so on (Mizuuchi et al., 2006). 



WM ^H If 1 I 




Fig. 1. Kotaro's photograph (left) and the arrangement of joints (right). 
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Musculoskeletal structure has advantages when realizing an articulated structure and 
installing mechanical flexibility. If an articulated structure consists of serial rotational 
joints actuated by rotational motors, then each motor has to generate torque enough to 
move or hold the mass and inertia of the descendant parts. In case of a tendon-driven 
structure, on the other hand, several actuators cooperatively work for moving one 
joint, as well as each actuator can determine the posture of one joint. A muscle-driven 
robot can have multi-articular muscles, which drive more than one joint. Human's 
body has several kinds of such muscles. 

Another advantage of musculoskeletal robots is easiness of installing mechanical elasticity 
and viscosity. Joints of muscle-driven skeletal robots are, in general, passive joints, and the 
structure around a joint is relatively simple. 

3. Design and Implementation of Kotaro 

Approximate height of Kotaro is 130 [cm], and weight is about 20 [kg] excluding power 
source and main computer which is placed outside the robot. In this section, Design and 
implementation of Kotaro is described. 

3.1 Arrangement of Joints 

The right figure of Fig.l shows the arrangement of Kotaro 7 s joints. There are 3-DOF ball- 
and-socket joints and 1-DOF rotational joints. We used ball-and-socket joints for most 
Kotaro's joints. Total degrees of freedom of joint angle space are 91. The spine consists of 5 
spherical joints, and the neck has 3. A leg has 8 DOF and an arm has 13. The structure of the 
shoulder, which is inspired from human's shoulder, consists of a collarbone and a 
bladebone, for expanding the movable range of the arm and increase inner space inside the 
chest (Sodeyama et al., 2005). Each four-finger hand has 11 DOF: 2 rotational joints for the 
thumb, and 3 rotational joints for the other fingers. Two color cameras are installed in the 
eye balls, which has three degrees of freedom; panning angles are independently 
controllable and tilting is synchronized. 

3.2 Reinforceable Muscle Humanoid 

Yet another advantage of musculoskeletal robots is configurability of muscles. We 
have proposed a concept of Reinforceable muscle humanoid (Mizuuchi et al., 2004, 
2005a). Joints are passive and we do not have to decide how to drive the joints and 
how much power to allocate at the design stage. It will be quite difficult to determine 
the actuator configurations beforehand, because the expected tasks for the robots in 
humans' life field are very broad. In case of a human or an animal, muscles are 
strengthened as growing up or by training. 

We have developed a new muscle unit which contains a 4.5W DC motor, pulleys, a tension 
sensor using strain gauges, an amplification circuit board and a thermometer. Fig.2 shows 
the unit. The black cords are chemical ropes. Fig.3 is a picture of a part of Kotaro 7 s body 
around hip joints, where some muscle units are attached. We can easily add/ remove units 
and modify the attaching positions of some units. There are many small holes on bones 
(structure parts) for attaching wires. All of higher power 20W motors and some of 4.5W 
motors to wind up the tendons are placed inside the bones as initial muscles, while others 
are reconfigurable muscle units. 



48 



Mobile Robots, Towards New Applications 




Fig. 2. A newly-developed, sensors-integrated muscle unit. 





Y * j 

Fig. 3. The muscle units attached around both crotch joints. 

3.3 Flexible Spine 

Human's spine with 24 joints has most degrees of freedom among human's body. We have 
assigned five spherical joints to Kotaro's spine. Each of upper three vertebrae has a rib bone 
(costa), which enlarges the moment arm and increase the torque around spine joints. In 
addition, rib bones make space inside the chest for installing motors and boards. Every vertebra 
has at least four points for attaching an end of a muscle-tendon. At least 4 tendons are needed to 
determine the posture of a spherical joint. There are 4 tendons between the pelvis and the 
lowest vertebra, and 4 between the pelvis and the second lowest vertebra. 4.5W motors installed 
on the pelvis pull these 8 muscle-tendons. There are 4 tendons between the lowest costa (third 
vertebra) and the middle costa (fourth vertebra). 4.5W motors installed in the lowest costa pull 
these 4 muscle-tendons. There are four main stomach muscles that are connecting between the 
pelvis and the middle costa (see Fig.4), pulled by four 20W motors in the pelvis. 
Kotaro's spine has physical softness. There is a part made of silicone rubber between every 
neighbor vertebrae. It is like human's interspinal disk, which is indicated as 8 in Fig.5. The 
figure shows the structure of human's spine (Kapandji, 1974), seen from the left side. The 
silicone rubber disks have elasticity and viscosity. Fig.6 is a photo of the pelvis and lower 
spine. In human's spine, there are ligaments (11 to 16 in Fig. 5) between the vertebral bones. 
Kotaro's spine has also tension springs between joints as ligaments. These rubbers and 
springs give the spine a force returning to the initial posture against gravity. This is also an 
effect of elasticity and helps the actuators. 
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Fig. 4. Bones (gray) and muscles (red) of Kotaro's torso. 




Fig. 5. Human's spine structure seen from left side (Kapandji, 1974). 

tension spring] 



silicone rubber 




Fig. 6. Elastic and viscous elements between joints. 



50 



Mobile Robots, Towards New Applications 



3.4 Collarbone and Bladebone 

Kotaro' s shoulder structure has been inspired from human's one. It consists of a collarbone and a 
bladebone. The advantage of this is the wide movable range and large space inside the chest 
(Sodeyama et al., 2005). Though the movable range of shoulder's spherical joint is not so wide, the 
center of rotation of the shoulder joint, which is on the shoulder blade, can be moved according to 
the movement of the bladebone and collarbone. The center of rotation of the movement of the 
shoulder joint is near the center of the body. The bladebone moves on the surface of the back chest. 
These bone structures are actuated by muscle-tendons. Fig.7 shows the shoulder structure of 
Kotaro. Fig.8 shows two postures of Kotaro' s left shoulder and arm. In arm-raising motion, 
human's bladebone start moving when the upper arm is raised to a certain extent. In the pictures, 
Kotaro is doing similar movement. 
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Fig. 7. Collarbone and bladebone. 




Fig. 8. A coupled movement of bladebone and upper arm. 



3.5 For Light and Strong Bones 

Though we use muscle units for reinforceability, some inevitable muscles are installed inside 
bones. Bones are needed to be tough and light, and possible to hold motors and circuit boards 
inside. Kotaro' s bones have been designed based on a concept namely 'hollowed mesh skeleton.' 
Fig.9 shows some bones; these are the pelvis, right thighbone, and so on. The radius of a bone is 
relatively large as a bone of an endoskeletal structure, in order to make space inside. The shapes of 
the holes of the mesh are designed so that the inner parts such as motors and circuit boards can be 
installed through the holes. The cross section of each mesh is T-shape or X-shape for increasing 
strength. Most bones are designed as single-piece parts. To form the structural parts, we used 
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some rapid prototyping (RP) methods. Main parts were made by selective laser sintering (SLS) 
method, and transparent parts were made by stereolithography. 




Fig. 9. Some // bones ,/ of Kotaro. 



3.6 Muscle-driven Head for Multi-modal Communication 

Fig. 10 shows Kotaro 7 s muscle-driven head for multi-modal communication with humans. 
In each eyeball, there is a USB2 color camera. Eyeballs are also tendon-driven. There are 
three actuators for eyes; the left-and-right movement is independent, and up-and-down is 
synchronized. There are two microphones in ears and one speaker near the mouth. These 
multimedia devices use USB, and are connected to a remote host PC through one USB cable. 
The same USB 480Mbps line is used for all the other sensors and actuators of the whole- 
body. There are also gyro sensors and accelerometers in the head. 




Fig. 10. Muscle-driven head for multi-modal communication. 



4. The Sensing System for Human Contact 

4.1 A Soft and Fleshy Tactile Sensor Using Conductive Rubber Foam 

We have developed a soft and fleshy tactile sensor using force-sensitive conductive rubber 
foam (Fig. 11). The black element in the figure is the rubber foam. There are electrodes 
inside the 3D shape. By measuring the resistance between electrodes, tactile information can 
be detected. In addition, there is a possibility of detecting the change of the 3D shape. Fig.ll 
shows the sensors. The right photo is Kotaro 7 s left hand; the palm and fingertips are made of 
the conductive rubber and there are electrodes inside. An advantage of this sensor is 
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flexibility of arranging the sensing electrodes. If more sensitivity is needed at an area, 
inserting an extra electrode can improve sensitivity. 




Fig. 11. Soft and fleshy tactile sensor using conductive rubber foam. 



4.2 A Free Form Tactile Sensor for Covering Whole Body 

For attaching tactile sensors on various shape surfaces of a robot, we propose a bandage- 
shape tactile sensor. This shape can match complex surface, compared with sheet-type 
tactile sensors. Fig. 12 shows the developed sensor. In the right photo, Kotaro's left arm is 
covered by this type of sensor. One bandage has 64 sensing points, and wiring forms an 8x8 
matrix. Connecting a bandage to a circuit board needs only a 16-line cable. Moreover, we 
can cut the bandage at any of cutting places for modifying the length without damaging the 
sensing circuit. This sensor consists of two thin films of flexible circuit boards and a thin film 
of force sensitive conductive rubber. When the rubber film is pushed, the resistance changes. 
Analog information of this sensor is also measured by using a small circuit board (Fig. 14). 




Fig. 12. Free form tactile sensor for covering whole body. 



4.3 Posture Sensor for a Ball-and-Socket Joint 

Joint angle sensor for a spherical joint has been a problem. We propose a new method for 
estimating the posture of a spherical joint by observing the ball by a small camera. On the 
surface of the ball, many spots are drawn. A visual processing unit connected to the camera 
tracks the movements of the spots and estimates the 3D joint-angle of the spherical joint. 
We implemented the sensing mechanism (Urata et al., 2006) by using a very small camera 
originally for mobile phones, and by using a processor (SH-mobile by Renesas; 10mm x 
10mm BGA) also originally for mobile phones. The right photo of Fig. 13 shows the 
developed boards, the cameral, and a coin. The size of the board is 1 inch A 2 (2.54 mm A 2). 
The lower left picture of Fig.13 shows a prototype of the ball, which we made at first. Inside 
of the socket is dark, so and LED was embedded in the ball. The upper left of Fig.13 shows 
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the schematic of the latest version, which uses and LED, plastic optical fibers, and some 
color filters for detecting the calibrating position. 




Fig. 13. Posture sensor for ball-and-socket joint using a mobile phone camera and a mobile 
phone microprocessor. 



4.4 Print Circuit Boards for Onbody Information Processing 

When the number of actuators and sensors in a robot is increased, it will be better to distribute 
the circuit boards for efficiency of wiring. We have designed Kotaro 7 s onboard system as a 
distributed system. We developed several kinds of circuit boards as shown in Fig. 14. The upper 
left photo shows motor driving boards for four 4.5W motors per a board, and the lower left one 
shows the boards for two 20 motors per a board. The size of both motor-driving boards is 36mm 
x 46mm. The upper right photo shows the boards that can collect 384 analog signals per one 
board (six 64x64 matrices). Every board of these three kinds has a USB1.1 interface (12Mbps). 
The lower right photo shows small USB2.0 (480Mbps) hub boards (compatible with USB1.1). 
The hub board has seven downstream ports and one upstream port, and it is able to connect 
commercial USB2 cameras with 640x480 pixels, USB microphones, USB speakers, and so on. All 
of the boards have been developed at the Kotaro project. There are about forty circuit boards in 
Kotaro 7 s body, and only one USB cable is connected to a remote host PC, which manages the 
whole body of the robot. 
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Fig. 14. Developed circuit boards (upper-left: for 4 small motors, lower-left: for 2 middle 
motors, upper-right: for 384 analog sensors, lower-right: 7-port USB2 hubs). 
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5. Summary and Conclusion 

This chapter has presented the concept and overview of Kotaro project, which aims at showing a 
proposal of robotics technologies of the year 2020. We joined the Prototype Robot Exhibition of the 
EXPO'05 held in Aichi, Japan, and performed demonstrations. Fig. 15 shows scenes at the 
exhibition. Kotaro has 91 DOF including flexible spine. Driving system of the endoskeletal 
structure is based on a wire-driven system which we call muscle-driven system. Currently it has 
about 90 motors and it can have up to 120 motors; sensor-integrated muscle units realize the easy 
addition of muscles. We call this characteristic as 'reinforceability.' Kotaro has physical flexibility 
by silicone rubber parts and tension springs embedded between joints, to achieve safety, supple 
motions, and human friendliness in the future. Kotaro has many sensors including muscle-tension 
sensors, muscle-length sensors, two kinds of tactile sensors, posture sensors for 3D spherical joints, 
two eyes, two ears, and so on. The skeletal structures are based on the concept 'hollowed mesh 
skeleton 7 and made by RP methods. 




Fig. 15. Demonstration scenes at the EXPO'05 in Aichi, Japan. 
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We believe that in future robots will be closer to human than present and their bodies 
should be much more compliant, and we may have to re-think about the structure of robot 
body fundamentally. We hope Kotaro will be a footstep to a new stage of humanoid 
robotics. Future problems include realizing various motions using the flexibility of the body, 
finding out a software system which can manage the complexity of input/ output of a robot, 
a framework of autonomous development like infants, and so on. Fig. 16 shows an ideal 
image of Kotaro: climbing a tree utilizing the flexible body. 




Fig. 16. An ideal image of Kotaro: climbing a tree utilizing the flexible body. 
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1. Introduction 

The first indications that the spinal marrow could contain the basic nervous system 
necessary to generate locomotion date back to the early 20th century. According to Mackay- 
Lyons (2002), the existence of nets of nervous cells that produce specific rhythmic 
movements for a great number of vertebrates is something unquestionable. Nervous nets in 
the spinal marrow are capable of producing rhythmic movements, such as swimming, 
jumping, and walking, when isolated from the brain and sensorial entrances. These 
specialised nervous systems are known as nervous oscillators or central pattern generators 
(CPGs). 

Grillner (1985), Collins & Stewart (1993), Pearson (1993), and Collins & Richmond (1994) are 
some interesting works about the locomotion of vertebrates controlled by central pattern 
generators. According to Moraes (1999), the relation between spinal marrow and encephalus 
in the central nervous system of domestic animals is most significant than relation in human 
beings. This occurs because the most motor activities in the animals is performed by reflexes 
and not by the cerebral activity. In relation to the total activity of the central nervous system, 
it is estimate that exist approximately ten times more activity in the spinal marrow of dogs 
than in humans. However, the human locomotion is controlled, in part, by a central pattern 
generator, which is evidenced in the works as Calancie et al. (1994), Dimitrijevic et al. (1998) 
and Pinter & Dimitrijevic (1999). 

Coupled nonlinear oscillators can be used in control systems of locomotion as pattern 
generators similar to the pattern of human gait, providing the approach trajectories of the 
legs. The central pattern generator is composed of a set of mutually coupled nonlinear 
oscillators, where each oscillator generates angular signals of reference for the movement of 
the legs. Each oscillator has its proper amplitude, frequency and parameters, and coupling 
terms makes the linking to the other oscillators. Some previous works on central pattern 
generators formed by nonlinear oscillators, applied in the locomotion of bipedal robots, can 
be seen in Bay & Hemami (1987), Dutra (1995), Zielinska (1996), Dutra et al. (2003) and Pina 
Filho etal. (2005). 

The objective of this chapter is to present the modelling of a bipedal robot using a central 
pattern generator formed by a set of coupled nonlinear oscillators. We present some 
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concepts about the central nervous system of human being, including its main structures 
and characteristics, as well as its relation with the motor functions. Although people do not 
move in completely identical way, there are some characteristics of the gait that can be 
considered universal, and these similar points serve as base for the description of the 
patterns involved in the locomotion. 

In the analyses, we consider a 2D model, with the three most important determinants of 
gait (the compass gait, the knee flexion of stance leg, and the plantar flexion of stance 
ankle), that performs only motions parallel to the sagittal plane. Using nonlinear 
oscillators with integer relation of frequency, we determine the transient motion and 
the stable limit cycles of the network formed by coupled oscillators, showing the 
behaviour of the hip angle and the knee angles. By changing a few parameters in the 
oscillators, modification of the step length and the frequency of the gait can be 
obtained. 

The study of the utilisation of this system in the locomotion has great application in the 
project of autonomous robots and in the rehabilitation technology, not only in the project of 
prosthesis and orthesis, but also in the searching of procedures that help to recuperate 
motor functions of human beings. 

2. The Central Nervous System 

According to Cordeiro (1996), in neurological point of view, the human locomotion is 
started by impulses of cerebral cortex for the voluntary control and fine coordination; to 
these stimulatons, are added the influences of the cerebellum (which becomes the 
coordinate gait), the vestibular system (which makes the maintenance of the balance), and 
the spinal marrow (which transmits the impulses to the motor organs through the 
peripheral nervous system, farther the maintenance of the posture). 

The nervous system is constituted by the central and peripheral systems. The peripheral 
nervous system is constituted by nerves and ganglia. The central nervous system is 
divided in two main parts: encephalus (brain cavity) and spinal marrow (vertebral 
canal). The encephalus is subdivided in cerebrum (formed by diencephalus and 
telencephalus), cerebellum and cerebral trunk or brainstem (formed by bulbus, pons 
and mesencephalus). Figure 1 shows in detail the division of the central nervous 
system. 

2.1 Cerebrum and Motor Cortex 

The cerebrum is formed by two main parts: diencephalus and telencephalus. Telencephalus 
is the name given to the cerebral hemispheres, predominant in the superior vertebrates, 
separate by the superior longitudinal fissure. The diencephalus functions as transmitter of 
the sensorial information to the cerebral cortex, contained a great number of neural circuits 
related to the vital functions, such as: regulation of the body temperature, cardiac frequency 
and arterial pressure. In relation to the locomotion, it is involved in the regulation of the 
posture and movement. 

The motor cortex is part of the cerebral cortex and its cells supply a direct canal between the 
cerebrum and the neurons of the spinal marrow, the so-called motoneurons. From studies 
with stimulation of the cortex performed by Penfield (1955), it was observed that the motor 
cortex is linked to all parts of the human body, including the inferior limbs, responsible for 
locomotion. 
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Fig. 1. Schema tical vision of the division of central nervous system. 

2.2 Cerebellum 

The cerebellum plays an important part in the regulation of the fine and complex 
movements, as well as in the temporal and spatial determination of activation of the muscles 
in the course of movement or in the postural adjustment (Brandao, 2004). 
In the locomotion movement, the cerebellum participates actively organising and updating, 
based in the analysis of sensorial information of position and speed of the limbs at every 
moment. This operation constitutes a planning at short term that overcoming the planning 
at long term performed by the cerebrum. 

This functional characteristic of the cerebellum explains the reason for which in the learning 
of a movement, the same one is performed at first, slowly, with intense mental 
concentration. With training and consequent motor learning, the amount of pre- 
programmed movements increases, resulting in a greater easiness and quickness in the 
performing of the same ones. 



2.3 Cerebral Trunk or Brainstem 

The cerebral trunk is formed by three main parts: mesencephalus, pons and bulbus. 
Mesencephalus is the portion more cranial of the cerebral trunk, crossed for the cerebral 
aqueduct. The pons functions as station for the information proceeding from the cerebral 
hemispheres, and are going to the cerebellum. The bulbus contains nuclei (groupings of 
bodies of neurons) and tratus (bundle of nervous fibres), which lead the sensorial 
information for the superior centers of the cerebrum, as well as nuclei and canals which lead 
motor commands of the cerebrum for spinal marrow. 

Parts of the central nervous system as pons, bulbus and spinal marrow, are endowed with 
programs of posture and movement, which are used by the organism when necessary, without 
the necessity of the involvement of regions located in a higher level in the central nervous system 
(Brandao, 2004). In fact, the main way of communication between the cerebrum and the rest of the 
organism is the spinal marrow, as it will be seen to follow. 
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2.4 Spinal Marrow 

The spinal marrow is formed by nerves that if extend from the cerebral cortex or in some 
areas of the cerebral trunk and finish, in its great majority, in the somatic motoneurons 
(which make connection with the muscles that performing the conscientious movements). It 
is protected by the bones of the spinal column and its nervous fibres pass through small 
openings between each vertebra. In the superior vertebrates, the spinal marrow is more 
strong subordinated to the cerebrum, executing its orders. 

The forepart of the spinal marrow contains the motor nerves (motoneurons), which transmit 
information to the muscles and stimulate the movement. The posterior part and the lateral parts 
contain the sensitive nerves, receiving information from the skin, joints, muscles and viscera. 
The human gait requires a coordination of the muscular activity between the two legs, 
which is made by a flexible neural coupling to the level of the spinal marrow (Dietz, 2003). 
Thus, in the course of the locomotion, a disturbance in one of the legs leads to a pattern of 
proposital reply of the spinal marrow, characterising the existence of the so-called central 
pattern generator. 

3. Locomotion Patterns 

The choice of an appropriate pattern of locomotion depends on the combination of a central 
programming and sensorial data, as well as of the instruction for one determined motor 
condition. This information determines the way of organisation of the muscular synergy, which 
is planned for adequate multiple conditions of posture and gait (Horak & Nashner, 1986). 



Sensorial information 




Central nervous Sensorial information 

system 

Fig. 2. Control system of the human locomotion. 

Figure 2 presents a scheme of the control system of the human locomotion, controlled by the 
central nervous system, which the central pattern generator supplies a series of pattern 
curves for each part of the locomotor. This information is passed to the muscles by means of 
a network of motoneurons, and the conjoined muscular activity performs the locomotion. 
Sensorial information about the conditions of the environment or some disturbance are 
supplied as feedback of the system, providing a fast action proceeding from the central 
pattern generator, which adapts the gait to the new situation. 
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Despite the people not walk in completely identical way, some characteristics in the gait can 
be considered universal, and these similar points serve as base for description of patterns of 
the kinematics, dynamics and muscular activity in the locomotion. 

In the study to be presented here, the greater interest is related to the patterns of the 
kinematics, in particular, of the hip and knee angles. From the knowledge of these patterns 
of behaviour, the simulation of the central pattern generator using the system of coupled 
oscillators becomes possible. 

3.1 Anatomical Planes of Movement 

In the study of the human movement, some definitions in relation to the planes of 
movement and regions of the body are adopted. The anatomical planes of movement are 
presented in Fig. 3, where: the sagittal plane divides the body in left side and right side; the 
frontal plane divides the body in a forepart region and posterior region; and the transverse 
plane divides the body in a superior part and another inferior. 




Transverse 

Plane 



Fig. 3. Anatomical planes of movement. 



3.2 The Articulation of the Hip 

The movements of the hip are performed in an only articulation. The hip is the articulation 
of the inferior limb close to the pelvis, being the joint that links this limb to the remaining 
portion of the body. This articulation performs the function of a spherical joint and is 
formed by the head of femur and acetabulum, located in the pelvis. 

The characteristics related to the amplitude of movement and stability are conditional for 
the functions of support of the weight and the locomotion, assumed for the inferior limb. 
These movements can be divided in movements of flexion or extension, aduction or 
abduction and external or internal rotation (Fig. 4). 

The flexion of the hip is the movement that leads the forepart of the thigh to the meeting of 
the trunk, while in the extension of the hip the contrary movement occurs. These 
movements are performed around a perpendicular axle to the sagittal plane of the joint. The 
configuration of the knee has direct influence on the amplitude of flexion of the hip. With 
the extension of the knee, the maximum angle of flexion of the hip is 90 degrees, while for 
the flexion of the knee, the amplitude reaches or exceeds 120 degrees. 
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Fig. 4. Movements of the hip. 

In the movement of abduction of the hip, the inferior limb is rotated directly for outside, 
moving away itself from the symmetry plane of the body, while the movement of aduction 
makes exactly the opposite. 

The movements of external and internal rotation of the hip are performed around of the 
mechanical axle of the inferior limb. In the normal position of alignment, this axle if 
confuses with the vertical axle of the joint, and the movement of external rotation rotate the 
foot for outside, while the movement of internal rotation rotate the foot for inside (in the 
direction of the symmetry plane of the body). 



3.3 Behaviour of the Hip in the course of Locomotion 

Considering the three anatomical planes of movement, from the use of an optic-electronic 
system of three-dimensional kinematical analysis (Raptopoulos, 2003), it is possible to define 
the angular behaviour of the hip in the course of the locomotion cycle. 

Figures 5 and 6 present the graphs of angular displacement and phase space of the hip in the 
sagittal plane, related to the movements of flexion and extension. Figures 7 and 8 are related to the 
movements of aduction and abduction in the frontal plane, while the Figures 9 and 10 show the 
behaviour of the hip to the movements of external and internal rotation in the transverse plane. 
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Fig. 5. Angular displacement of the hip in the sagittal plane (mean ± deviation). 
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Fig. 6. Phase space of the hip in the sagittal plane. 
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Fig. 7. Angular displacement of the hip in the frontal plane (mean ± deviation). 
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Fig. 8. Phase space of the hip in the frontal plane. 
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Fig. 9. Angular displacement of the hip in the transverse plane (mean ± deviation). 
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Fig. 10. Phase space of the hip in the transverse plane. 



3.4 The Articulation of the Knee 

The knee is the intermediate articulation of the inferior limb, being the joint that links femur 
to the tibia. This articulation have two degrees of freedom, being the flexion-extension the 
main movement, and the rotation around of the longitudinal axle of the leg is a auxiliary 
movement, which only appears when the knee is bent. This articulation is classified as a 
condilar joint, where a main movement occurs in the sagittal plane (movement of flexion 
and extension) and another one, secondary, occurs as composition of the rotation and 
translation of the segment of the tibia. Due to the restrictions imposed for the ligaments of 
the knee and the proper condilar surface, this does not present great amplitude of 
movement in the frontal and transverse planes in the course of the support phase. Figure 11 
shows the movements of the knee in each one of the anatomical planes. 
The flexion of the knee is a movement that approaches the posterior region of the leg to the 
posterior side of the thigh. The amplitude can come at 140 degrees with the hip in flexion, 
but it does not exceed 120 degrees with the hip in extension. The extension of the knee is 
defined as the movement that moves away the posterior side of the leg to the posterior side 
of the thigh. When the extension is extreme, this receives the name of genu recurvatum and 
has pathological causes. 



Modelling of Bipedal Robots Using Coupled Nonlinear Oscillators 



65 





V-fcii 




Fig. 11. Movements of the knee. 

The knee presents lateral displacements that depend on the sex and pathological variations. 

When the knee is dislocated laterally, says that the articulation is in genu varo. When the 

knee is dislocated medially, says that the articulation is in genu valgo. 

The movement of axial rotation of the knee occurs around of its longitudinal axle and only it can 

be performed in flexion, therefore in extension the knee is braked. This braking mechanism is 

related to the articulate surfaces and has the function to increase the resistance of the knee to the 

torque that occurs in the course of the phase of contact of the limb with the ground. 

More details about the physiology of the articulations of the knee and other parts of the 

body can be seen in Kapandji (1980). 



3.5 Behaviour of the Knee in the course of Locomotion 

In similar way that performed for the hip, considering the three anatomical planes of 
movement, from the use of an optic-electronic system of three-dimensional kinematical 
analysis (Raptopoulos, 2003), it was possible to define the angular behaviour of the knee in 
the course of the locomotion cycle. 

Figures 12 and 13 present the graphs of angular displacement and phase space of the knee in the 
sagittal plane, related to the movements of flexion and extension. Figures 14 and 15 are related to 
the movements of genu varo and genu valgo in the frontal plane, while Figures 16 and 17 show the 
behaviour of the knee to the movements of external and internal rotation in the transverse plane. 
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Fig. 12. Angular displacement of the knee in the sagittal plane (mean ± deviation). 
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Fig. 13. Phase space of the knee in the sagittal plane. 
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Fig. 14. Angular displacement of the knee in the frontal plane (mean ± deviation). 
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Fig. 15. Phase space of the knee in the frontal plane. 
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Fig. 16. Angular displacement of the knee in the transverse plane (mean ± deviation). 
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Fig. 17. Phase space of the knee in the transverse plane. 



4. Mechanical Model 

Before a model of central pattern generator can be applied to a physical system, the desired 
characteristics of the system must be completely determined, such as the movement of the 
leg or another rhythmic behaviour of the locomotor. Some works with description of the 
rhythmic movement of animals include Eberhart (1976), Winter (1983) and McMahon (1984), 
this last one presenting an ample study about the particularities of the human locomotion. 
To specify the model to be studied is important to know some concepts related to the 
bipedal gait, such as the determinants of gait. 

The modelling of natural biped locomotion is made more feasible by reducing the number of 
degrees of freedom, since there are more than 200 degrees of freedom involved in legged 
locomotion. According to Saunders et al. (1953) the most important determinants of gait are: 1) the 
compass gait that is performed with stiff legs like an inverted pendulum. The pathway of the 
center of gravity is a series of arcs; 2) pelvic rotation about a vertical axis. The influence of this 
determinant flattens the arcs of the pathway of the center of gravity; 3) pelvic tilt, the effects on the 
non-weight-bearing side further flatten the arc of translation of the center of gravity; 4) knee 
flexion of the stance leg. The effects of this determinant combined with pelvic rotation and pelvic 
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tilt achieve minimal vertical displacement of the center of gravity; 5) plantar flexion of the stance 
ankle. The effects of the arcs of foot and knee rotation smooth out the abrupt inflexions at the 
intersection of the arcs of translation of the center of gravity; 6) lateral displacement of the pelvis. 




Fig. 18. Three-dimensional model with the six most important determinants of gait. 

In order to perform these determinants of gait, a 3D model with 15 degrees of freedom (Fig. 
18) is needed. The kinematical analysis, using the characteristic pair of joints method is 
presented in Saunders et al. (1953). 

In order to simplify the investigation, a 2D model that performs motions parallel only to the sagittal 
plane will be considered. This model, depicted in Fig. 19, characterises the three most important 
determinants of gait, determinants 1 (the compass gait), 4 (knee flexion of the stance leg), and 5 
(plantar flexion of the stance ankle). The model does not take into account the motion of the joints 
necessary for the lateral displacement of the pelvis, for the pelvic rotation, and for the pelvic tilt. 




Fig. 19. 2D model with the most important determinants of gait and relative angles. 

This model must be capable to show clearly the phenomena occurred in the course of the motion. 
The adopted model works with the hypothesis of the rigid body, where the natural structural 
movements of the skin and muscles, as well as bone deformities, are disregarded. The locomotion 
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cycle can be divided in two intervals: single support phase and double support phase. In single 
support phase, one of the legs performs the swinging movement, while the another one it is 
responsible for the support. The extremity of the support leg is assumed as not sliding. The double 
support phase is the phase where the transition of the legs occurs, the swinging leg becomes 
supporting leg and to another one it gets ready to initiate the swinging movement. This phase is 
initiated at the moment where the swinging leg touches the ground. 

Considering the 2D model adopted in the Fig. 19, the knee angles 63 and 612, and the hip 
angle 69, will be determined by a system of coupled nonlinear oscillators. Such oscillators 
must present an asymptotic behaviour, typical of the dissipative systems, and characterised 
for the existence of attractors, more specifically limit cycles, as it will be seen to follow. 

5. Nonlinear Oscillators 

Oscillators are used to describe mechanisms that repeat its action periodically, such as: some 
neurons, electric circuits, waves, cells, etc. The behaviour of an oscillator can be described by 
a differential equation, whose solution presents cyclical behaviour. Thus, nonlinear 
oscillators can be represented by nonlinear differential equations. 

A question of particular interest in nonlinear systems is the existence of closed trajectories, which 
imply in periodic movement. Closed trajectories occur not only in conservative systems, but also 
in non-conservative systems, however in such systems, in the end of a cycle, the change of energy 
must be zero. This implies that during parts of the cycle the energy is wasted, and during other 
parts the system receives energy, what it keeps the balance of energy in zero. The closed 
trajectories in this in case receive the name of limit cycles. Limit cycles can be considered as 
movements of balance in which the system performs periodic movement (Meirovitch, 1975). 
An important characteristic of the oscillators with limit cycles is that, while the amplitude of a 
closed trajectory for a conservative system depends initially on the energy supplied to the system, 
the amplitude of a limit cycle not only depends on the energy, but of the system parameters. One 
of the oscillators with limit cycle more known and used in diverse works about locomotion is the 
van der Pol oscillator. Another similar oscillator to the van der Pol, the Rayleigh oscillator, is less 
known and it was little explored, as well as its application in the locomotion. The system of 
coupled oscillators proposed in this work uses these Rayleigh oscillators. 

5.1 Van der Pol Oscillator 

Balthazar van der Pol (1889-1959) was a Dutch engineer who made dynamic experimental 
studies in the beginning of 20th century. He investigated electric circuits using vacuum 
tubes and verified that the circuits presented stable oscillations (limit cycles). Working with 
van der Mark, van der Pol was one of the first ones to present an article with experimental 
studies on chaos. He also performed diverse studies about the human heart, building 
models with the objective to study the dynamic stability of same one. 

The equation of van der Pol is wellknown, originated from a simple circuit RLC in which 
was inserted an active nonlinear element. The form most common of the equation of van der 
Pol is given by the equation: 

x + ju(x 2 -1)jc + jc = (1) 

where ju is the damping term. More details and discussions about the equation of van der 
Pol, also considering forcing terms, can be seen in Jackson (1990) and Strogatz (1994). 
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5.2 Rayleigh Oscillator 

In the course of 19th century, some works related with nonlinear oscillators had been 
performed, particularly in conjunction with models of musical instruments. At this time, the 
British mathematical physicist Lord Rayleigh (John William Strutt, 1842-1919) introduced an 
equation of the form: 



mx" + kx = ax- b\x) 



(2) 



(with nonlinear damping speed) to simulate the oscillations of a clarinet. The equation of 
Rayleigh used in the analyses will be: 



x-S\i-qx 2 )x + £l 2 (x-x ) = 8,q>0 



(3) 



where 8, q and Q correspond to the parameters of the oscillator. 

Despite the apparent similarity, the oscillators of van der Pol and Rayleigh present distinct 
behaviours. In electric point of view, the oscillators answer to an increase of voltage of different 
form. In the case of the van der Pol oscillator, an increase of the voltage implies in increase of the 
frequency, while in the Rayleigh oscillator it implies in an increase of the amplitude. 
Considering y = x , we have the following autonomous system: 



--y 



y- 



s(l-qy 2 )y-Q 2 



-*oJ 



(4) 



Choosing values for 8, q, £1 and xo, by means of a program that integrates the ordinary 
differential equations (ODE), is possible to plot the graphs of x and x as function of the time 
and the trajectory in the phase space. 



.x, r 




Fig. 20. Graph of x and X as function of the time. 

Making 8 = q = Q = 1 and xo = 0, and using the MATLAB®, the graphs for the Rayleigh 
oscillator had been generated, presented in Figures 20, 21 and 22. It observes the formation 
of the limit cycle (Fig. 21), that it implies in periodic movement that can be seen in Fig. 20. 
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Fig. 21. Trajectory in the phase space (limit cycle). 
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Fig. 22. Phase portrait. 

5.3 Coupled Oscillators System 

Oscillators are said coupled if they allow themselves to interact, in some way, one with the 
other, as for example, a neuron that can send a signal for another one in regular intervals. 
Mathematically speaking, the differential equations of the oscillators have coupling terms 
that represent as each oscillator interacts with the others. 

According to Kozlowski et al. (1995), since the types of oscillators, the type and topology of 
coupling, and the external disturbances can be different, exist a great variety of couplings. 
In relation to the type of coupling, considering a set of n oscillators, exists three possible 
basic schemes (Low & Reinhall, 2001): 1) coupling of each oscillator to the closest 
neighbours, forming a ring (with the n-th oscillator coupled to the first one): 



i = !..», 



J = \ 



i + l,n i = \ 

/ + l,i-l i = 2..n-l 

1, i — 1 i — n 



(5) 
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2) coupling of each oscillator to the closest neighbours, forming a chain (with the n-th 
oscillator not coupled to the first one): 



(6) 



3) coupling of each one of the oscillators to all others (from there the term "mutually coupled"): 

i = \..n, j = \..n, j ^ i (7) 

This last configuration of coupling will be used in the analyses, since it desires that each one of the 
oscillators have influence on the others. Figure 23 presents the three basic schemes of coupling. 

0=0 /- °""™ 

1 'J o^o^o 



0=0 




Cunning, 



(■J ft} ft) 

Fig. 23. Basic schemes of coupling: in ring (a), in chain (b) and mutually coupled (c). 

5.4 Coupled Oscillators with the Same Frequency 

From the equation (3), considering a net of n-coupled Rayleigh oscillators, and adding a 
coupling term that relates the velocities of the oscillators, we have: 



9 i -d i ^-q i ef)9 i +Q} i {9 i - 



-2>vM;) 



i=l,2, ...,« 



(8) 



7=1 



where 5 Z , qi, Q; and c Z/ y are the parameters of this system. 



For small values of parameters determining the model nonlinearity, we will assume that the 
response is approximated by low frequency components from full range of harmonic 
response. Therefore periodic solutions can be expected, which can be approximated by: 



Q t = 6 io + A t cos(a)t + a t ) 



(9) 



In this case, all oscillators have the same frequency co. Deriving the equation (9) and 
inserting the solutions in (8), by the method of harmonic balance (Nayfeh and Mook, 1979), 
the following system of nonlinear algebraic equations are obtained: 



(10) 



A i \p.j -co 2 J cos a i + AjSjCol 1 l — L sin a t + o)S\ c tj [A sin a i ~ A j sin a j )= 

Ajlfo 2 -Q^jsincTj- 4-^-^flJ 1 — cos^- +co\\c i j(A i cos ex t - AjCoscXj)=0 

With this system of equations, the parameters c\\ and Q; can be calculated: 



3a) 2 A? 3o) 2 AfS t j^ J J J 



(11) 
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Q/ = \r ~T^ AjCiJ sin ^' ~ aj ^ 



(12) 



Given the amplitude A t and A ■ , phase oc; and Oj, the frequency co, and the chosen values of 
5; and a t j, the value of the parameters c\\ and Q; can be calculated. 

5.5 Coupled Oscillators with Integer Relation of Frequency 

Oscillators of a coupling system, with frequency co, can be synchronised with other 
oscillators with frequency nco, where n is an integer. In the study of human locomotion, we 
can observe that some degrees of freedom have twice the frequency of the others (n = 2). 
Therefore, a net of coupled Rayleigh oscillators can be described as: 



e h -s h ^-q h el)e h +a 2 {e h -eJ-J^c^fa - 






(13) 



where the term c hi [0 t (0 t - io )\ is responsible for the coupling between two oscillators with 
different frequencies, while the other term c hk \6 h - 6 k ) makes the coupling between two 

oscillators with the same frequencies. 

If the model nonlinearity is determined for small values of parameters, periodic solutions 

can be expected which can be approximated by the harmonic functions: 

h = ho + A h cos(2o)t + a h ) (14) 

6 i - 6 io + A; cos(cot + a t ) (15) 

6 k = 6 ko + A k cos(2cot + a k ) (16) 

Deriving the equation (14-16) and inserting the solutions in (13), by the method of 
harmonic balance (Nayfeh and Mook, 1979), the following system of nonlinear 
algebraic equations are obtained: 



A h \fl 2 h - 4co 2 Jcosa h + 2A h S h a>\l-3a) 2 A%q h )sina h +co\\— ! —c hi sin2ar ; - 

i=i 

n 

2coy c hk (A h sin a h - A k sin a k ) = 
k=i 

A h [4o) 2 -^Jsinc^ +2A h S h o)\l-3o) 2 Aj;q h Jcosa h +Q)S\— l —c hi cos2a i 



(17) 



loo 2_^ c hk (A h cos a h - A k cos a k ) 



:0 



With this system of equations, the parameters q^ and Q^ can be obtained: 

1 1 m 

— r~> r + — r^ — ^ A hh i cos fe - 2a i ) + 
3co 2 A 2 h \2a> 2 Al8 h *-t 



<lh : - 



(18) 



J— J] C Kk Uh ~ A k C0S K - a k )] 



3co 2 A\8 h -^ 



74 Mobile Robots, Towards New Applications 



Q h = Uco 2 +^YAfc hJ sin{a h -2a i )--^YA k c htk sm(a h -a k ) v > 

Given the amplitude ^ , A i and ^ , phase Ofa, oa and e&, the frequency co, and the chosen 
values of Sh, Ch,i and Ch,h the value of the parameters q% and £1% can be calculated. 



6. Analysis and Results of the Coupling System 

To generate the motion of knee angles 63 and 612, and the hip angle 69, as a periodic attractor of a 
nonlinear network, a set of three coupled oscillators had been used. These oscillators are mutually 
coupled by terms that determine the influence of each oscillator on the others (Fig. 24). How much 
lesser the value of these coupling terms, more "weak" is the relation between the oscillators. 




MO *i.n Mil j 



Fig. 24. Structure of coupling between the oscillators. 

Considering Fig. 24, from the Equation (13) the coupling can be described for the equations: 

#3 -s^-q 3 e^)e 3 +n 3 2 fe -Oyo)-c^[e 9 {e 9 -e 9o ]\-c xn {e 3 -o l2 )=o (20) 

e 9 -8 9 ^-q 9 ei]e 9 +nl{o 9 -e 9o )-c %3 [e 3 (e 3 -e 3o )\- c%xl \e xl [e xl -e Uo )\ = o (21) 

#12 "^(l-^^)^^ ( 22 ) 

The synchronised harmonic functions, corresponding to the desired movements, can be writing as: 

<9 3 = 6 3o + A 3 cos(2cot + a 3 ) (23) 

<9 9 = A 9 cos(wt + a 9 ) (24) 

6 n = 6\2o + A n cos(2cot + a u ) (25) 

Considering a 3 = 0(9 = oc\i - and deriving the equation (23-25), inserting the solution into 
the differential equations (20-22), the necessary parameters of the oscillators (c\i and Q,> i e 
{3, 9, 12}) can be determined. Then: 

^3,12(^3 -Ai) + ^ 3 8 3 +j4$c 3t9 (r)f . 

q 3 = ^~r- \ Zb ) 

(27) 
(28) 



\2a) 2 A 3 3 S 3 


£l 3 =20) 


4 


3co 2 A 9 2 
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?12 : 



Q 9 =Q) 

_ 4c U j{A U - A 3) + 4A 12^2+ A 9 C 12,9 

\2o) 2 A^ 2 S u 
Q 12 =2co 



(29) 
(30) 

(31) 



From equations (20-22) and (26-31), and using the MATLAB®, the graphs shown in Fig. 25 

and 26 were generated, and present, respectively, the behaviour of the angles as function of 

the time and the stable limit cycles of the oscillators. 

These results were obtained by using the parameters showed in Table 1, as well as the initial 

values provided by Table 2. All values were experimentally determined. 

In the Fig. 26, the great merit of this system can be observed, if an impact occurs and the 

angle of one joint is not the correct or desired, it returns in a small number of periods to the 

desired trajectory. Considering, for example, a frequency equal to 1 s _1 , with the locomotor 

leaving of the repose with arbitrary initial values: 63 = -3°, 69 = 40° and 612 = 3°, after some 

cycles we have: 6 3 = 3°, 6 9 = 50° and 612 = -3°. 
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Fig. 25. Behaviour of 63, 69 and 612 as function of the time. 
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Fig. 26. Trajectories in the phase space (stable limit cycles). 
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C3,9 


C9,3 


C3,12 


Cl2,3 


C9,12 


Cl2,9 


£3 


£9 


£\2 


0.001 


0.001 


0.1 


0.1 


0.001 


0.001 


0.01 


0.1 


0.01 



Table 1. Parameters of Rayleigh oscillators. 



Cycle 


A 3 


A 9 


A 12 


fto 


fto 


#L2o 


0< at<n 


-29 


50 


10 


32 





-13 


n< wt<2n 


-10 


50 


29 


13 





-32 



Table 2. Experimental initial values. 

Comparing Fig. 25 and 26 with the experimental results presented in Section 3 (Fig. 5, 6, 12, 
13), it is verified that the coupling system supplies similar results, what confirms the 
possibility of use of mutually coupled Rayleigh oscillators in the modelling of the CPG. 
Figure 27 shows, with a stick figure, the gait with a step length of 0.63 m. Figure 28 shows 
the gait with a step length of 0.38 m. Dimensions adopted for the model can be seen in Table 
3. More details about the application of coupled nonlinear oscillators in the locomotion of a 
bipedal robot can be seen in Pina Filho (2005). 





Toes 


Foot 


Leg (below the knee) 


Thigh 


Length [m] 


0.03 


0.11 


0.37 


0.37 



Table 3. Model dimensions. 
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Fig. 27. Stick figure showing the gait with a step length of 0.63 m. 
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Fig. 28. Stick figure showing the gait with a step length of 0.38 m. 
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7. Conclusion 

From presented results and their analysis and discussion, we come to the following 
conclusions about the modelling of a bipedal locomotor using mutually coupled 
oscillators: 1) The use of mutually coupled Rayleigh oscillators can represent an excellent 
way to signal generation, allowing their application for feedback control of a walking 
machine by synchronisation and coordination of the lower extremities. 2) The model is 
able to characterise three of the six most important determinants of human gait. 3) By 
changing a few parameters in the oscillators, modification of the step length and the 
frequency of the gait can be obtained. The gait frequency can be modified by means of the 
equations (23-25), by choosing a new value for co. The step length can be modified by 
changing the angles 69 and 612, being the parameters c\\ and Q,-, i e {3, 9, 12}, responsible 
for the gait transitions. 

In future works, it is intended to study the behaviour of the ankles, as well as simulate the 
behaviour of the hip and knees in the other anatomical planes, thus increasing the network 
of coupled oscillators, looking for to characterise all determinants of gait, and consequently 
simulate with more details the central pattern generator of the human locomotion. 
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The Zero Moment Point (ZMP) and Centroidal Moment Pivot (CMP) are important 
ground reference points used for motion identification and control in biomechanics 
and legged robotics. Using a consistent mathematical notation, we define and compare 
the ground reference points. We outline the various methodologies that can be 
employed in their estimation. Subsequently, we analyze the ZMP and CMP trajectories 
for level-ground, steady-state human walking. We conclude the chapter with a 
discussion of the significance of the ground reference points to legged robotic control 
systems. In the Appendix, we prove the equivalence of the ZMP and the center of 
pressure for horizontal ground surfaces, and their uniqueness for more complex 
contact topologies. 

Since spin angular momentum has been shown to remain small throughout the walking 
cycle, we hypothesize that the CMP will never leave the ground support base throughout 
the entire gait cycle, closely tracking the ZMP. We test this hypothesis using a 
morphologically realistic human model and kinetic and kinematic gait data measured from 
ten human subjects walking at self-selected speeds. We find that the CMP never leaves the 
ground support base, and the mean separation distance between the CMP and ZMP is small 
(14% of foot length), highlighting how closely the human body regulates spin angular 
momentum in level ground walking. 

KEY WORDS— Legged Locomotion, Control, Biomechanics, Human, Zero Moment Point, 
Center of Pressure, Centroidal Moment Pivot 

1. Introduction 

Legged robotics has witnessed many impressive advances in the last several decades— from 
animal-like, hopping robots in the eighties (Raibert 1986) to walking humanoid robots at 
turn of the century (Hirai 1997; Hirai et al. 1998; Yamaguchi et al. 1999; Chew, Pratt, and 
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Pratt 1999; Kagami et al. 2000). Although the field has witnessed tremendous progress, 
legged machines that demonstrate biologically realistic movement patterns and behaviors 
have not yet been offered due in part to limitations in control technique (Schaal 1999; Pratt 
2002). An example is the Honda Robot, a remarkable autonomous humanoid that walks 
across level surfaces and ascends and descends stairs (Hirai 1997; Hirai et al.1998). The 
stability of the robot is obtained using a control design that requires the robot to accurately 
track precisely calculated joint trajectories. In distinction, for many movement tasks, animals 
and humans control limb impedance, allowing for a more robust handling of unexpected 
disturbances (Pratt 2002). 

The development of animal-like and human-like robots that mimic the kinematics and 
kinetics of their biological counterparts, quantitatively or qualitatively, is indeed a 
formidable task. Humans, for example, are capable of performing numerous dynamical 
movements in a wide variety of complex and novel environments while robustly rejecting 
a large spectrum of disturbances. Given limitations on computational capacity, real-time 
trajectory planning in joint space does not seem feasible using optimization strategies 
with moderately-long future time horizons. Subsequently, for the diversity of biological 
motor tasks to be represented in a robot's movement repertoire, the control problem has 
to be restated using a lower dimensional representation (Full and Koditschek 1999). 
However, independent of the specific architecture that achieves that reduction in 
dimension, biomechanical motion characteristics have to be identified and appropriately 
addressed. 

There are two ground reference points used for motion identification and control in 
biomechanics and legged robotics. The locations of these reference points relative to each 
other, and relative to the ground support area, provide important local and sometimes 
global characteristics of whole-body movement, serving as benchmarks for either physical 
or desired movement patterns. The Zero Moment Point (ZMP), first discussed by Elftman 1 
(1938) for the study of human biomechanics, has only more recently been used in the 
context of legged machine control (Vukobratovic and Juricic 1969; Vukobratovic and 
Stepanenko 1972; Takanishi et al. 1985; Yamaguchi, Takanishi and Kato 1993; Hirai 1997; 
Hirai et al. 1998). The Centroidal Moment Pivot (CMP) is yet another ground reference point 
recently introduced in the literature (Herr, Hofmann, and Popovic 2003; Hofmann, 2003; 
Popovic, Hofmann, and Herr 2004a; Goswami and Kallem 2004). When the CMP 
corresponds with the ZMP, the ground reaction force passes directly through the CM of the 
body, satisfying a zero moment or rotational equilibrium condition. Hence, the departure of 
the CMP from the ZMP is an indication of non-zero CM body moments, causing variations 
in whole-body, spin angular momentum. 

In addition to these two standard reference points, Goswami (1999) introduced the Foot 
Rotation Indicator (FRI), a ground reference point that provides information on stance-foot 
angular accelerations when only one foot is on the ground. However, recent study (Popovic, 
Goswami and Herr 2005) find that the mean separation distance between the FRI and ZMP 
during the powered plantar flexion period of single support is within the accuracy of their 



1 Although Borelli (1680) discussed the concept of the ZMP for the case of static equilibrium, 
it was Elftman (1938) who introduced the point for the more general dynamic case. Elftman 
named the specified point the "position of the force" and built the first ground force plate 
for its measurement. 
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measurement (0.1% of foot length), and thus the FRI point is determined not to be an 
adequate measure of foot rotational acceleration. 

In this chapter we study the ZMP and CMP ground reference points. Using a consistent 
mathematical notation, we define and compare the ground points in Section 2.0 and outline 
the various methodologies that can be employed in their estimation. In Section 3.0, we 
analyze the ZMP and CMP trajectories for level-ground, steady-state human walking, and 
in Section 4.0, we conclude the paper with a discussion of the significance of the ground 
reference points to legged robotic control systems. 

In Section 3.0, key hypothesis is tested regarding the nature of the ground reference points 
in level-ground, steady-state human walking. Because recent biomechanical investigations 
have shown that total spin angular momentum is highly regulated throughout the walking 
cycle (Popovic, Gu, and Herr 2002; Gu 2003; Herr, Whiteley and Childress 2003; Popovic, 
Hofmann, and Herr 2004a; Herr and Popovic 2004), we hypothesize that the CMP trajectory 
will never leave the ground support base throughout the entire walking gait cycle, closely 
tracking the ZMP trajectory throughout the single and double support phases of gait. We 
test both the CMP hypothesis using a morphologically realistic human model and kinetic 
and kinematic gait data measured from ten human subjects walking at self -selected walking 
speeds. 

2. ZMP and CMP Reference Points: Definitions and Comparisons 

In this section, we define the ground reference points: ZMP and CMP. Although the 
reference points have been defined previously in the literature, we define and compare 
them here using a consistent terminology and mathematical notation. 

In this paper, we adopt a notation by which r(r A ) symbolizes the total moment acting on 
a body about point r A ■ For example, f (0) symbolizes a moment calculated at the origin of 
a coordinate frame. This notation stresses the fact that a moment of force acting on a body 
changes depending on the point about which it is calculated. In addition to the point 
about which the moment is calculated, we also designate the force used in the moment 
calculation. For example, if we consider only the moment due to the ground reaction force 
acting on a body, we specify this with the subscript G.R., i.e. ? GR (p A ). Also, in this paper 

when we consider only a moment that acts on a particular body segment, or group of 
segments, we specify that moment using the segment's name in the superscript, e.g. 
r foot (r A )- In addition, in this manuscript, we often refer to the ground support base (GSB) 
to describe the foot-ground interaction. The GSB is the actual foot-ground contact surface 
when only one foot is in contact with the ground, or the convex hull of the two or more 
discrete contact surfaces when two or more feet are in contact with the ground, 
respectively. Finally, the ground support envelope is used to denote the actual boundary 
of the foot when the entire foot is flat on the contact surface, or the actual boundary of the 
convex hull when two or more feet are flat on the contact surface. In contrast to the 
ground support base, the ground support envelope is not time varying even in the 
presence of foot rotational accelerations or rolling. 

2.1 Zero Moment Point (ZMP) 

In the book On the Movement of Animals, Borelli (1680) discussed a biomechanical point 
that he called the support point, a ground reference location where the resultant 
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ground reaction force acts in the case of static equilibrium. Much later, Elftman (1938) 
defined a more general "position of the force" for both static and dynamic cases, and 
he built the first ground force plate for its measurement. Following this work, 
Vukobratovic and Juricic (1972) revisited Elftman's point and expanded its definition 
and applicability to legged machine control. They defined how the point can be 
computed from legged system state and mass distribution, allowing a robotic control 
system to anticipate future ground-foot interactions from desired body kinematics. For 
the application of robotic control, they renamed Elftman's point the Zero Moment 
Point (ZMP). 

Although for flat horizontal ground surfaces the ZMP is equal to the center of pressure, the 
points are distinct for irregular ground surfaces. In the Appendix of this manuscript, we 
properly define these ground points, and prove their equivalence for horizontal ground 
surfaces, and their uniqueness for more complex contact topologies. 

Vukobratovic and Juricic (1969) defined the ZMP as the "point of resulting reaction forces at 
the contact surface between the extremity and the ground". The ZMP, r ZMP , therefore may 
be defined as the point on the ground surface about which the horizontal component of the 
moment of ground reaction force is zero (Arakawa and Fukuda 1997; Vukobratovic and 
Borovac 2004), or 

T G.R. ( r ZMP ) \horizontal = ° CO 

Equation (1) means that the resulting moment of force exerted from the ground on the body 
about the ZMP is always vertical, or parallel to g . The ZMP may also be defined as the 

point on the ground at which the net moment due to inertial and gravitational forces has no 
component along the horizontal axes (Hirai et al. 1998; Dasgupta and Nakamura 1999; 
Vukobratovic and Borovac 2004), or 

^ inertia + gravity \'ZMP ) I horizontal ~ ^ • \ ) 

Proof that these two definitions are in fact equal may be found in Goswami (1999) and more 
recently in Sardain and Bessonet (2004). 

Following from equation (1), if there are no external forces except the ground reaction force 
and gravity, the horizontal component of the moment that gravity creates about the ZMP is 
equal to the horizontal component of the total body moment about the ZMP, 

*(?ZMP) \horizontal ' or 
^ VZMP) \horizontal = IV CM ~ ^ZMP) x ^g \horizontal @) 

where Yq M is the center of mass (CM) and M is the total body mass. Using detailed 
information of body segment dynamics, this can be rewritten as 



\ r i- r ZMp) x ^i^ + d 



= \{r C M ~ r Z Mp)>< ^horizontal ( 4 ) 

horizontal 



where r t is the CM of the i -th link, w z - is the mass of the i -th link, a i is the linear acceleration 
of the i -th link CM, I t is the inertia tensor of the i -th link about the link's CM, and <^ is the 



Ground Reference Points in Legged Locomotion: 
Definitions, Biological Trajectories and Control Implications 



83 



angular velocity of the i -th link. Equation (4) is a system of two equations with two 
unknowns, x ZMP and yzMP > * na t can be solved to give 
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Given full body kinematics and the mass distribution of a legged system, equation (5) 
can be used to reconstruct the ZMP trajectory. Alternatively, at a particular instant in 
time, equation (5) can be employed as a constraint equation for deciding joint 
accelerations consistent with a desired ZMP position, as discussed by Kondak and 
Hommel (2003). 

Finally, the ZMP as a function of the CM position, net CM force ( F = Ma CM ), and net 
moment about the CM can be expressed as 



X ZMP ~ X CM ' 



v VCM ) 



Z CM 

F z +Mg F z +Mg 



and 



(6) 



yzMP - ycM ■ 



F z +Mg 



Z CM + 



F z +Mg 



As emphasized in Figure 1, the most important notion of the ZMP quantity, applicable for 
both single and multi-leg ground support phases, is that it resolves the ground reaction 
force distribution to a single point. However, one needs to be careful to use this point in an 
appropriate manner. Most notably, both the vertical component of moment and the CM 
work performed by the ground reaction force cannot be computed solely on the bases of the 
ZMP trajectory and the resulting ground reaction force vector. For example, the resultant 
horizontal ground reaction force could be zero while the vertical component of moment 
and/ or the work performed by the ground reaction force could be nonzero. Consider a 
legged posture in which the following conditions are satisfied: 1) the ZMP is located just 
beneath the CM; 2) the horizontal ground reaction force field is tangent to a circle centered 
about the ZMP; and 3) the horizontal ground reaction force magnitude is a function of only 
radial distance. In this situation, shown in Figure 2, the net horizontal force is zero, but the 
net moment is nonzero. Another example is two particles of equal mass subject to two forces 
equal in magnitude but acting in opposite directions; while the net force is zero and the CM 
is at rest, the particles are moving and the work conducted by the external forces is nonzero. 

In other words, neither SWg.r. = ^g.r^zmp nor $Wg.r. = Fq.r&cm are permissible 
expressions for the work performed by the ground reaction force. 
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Fig. 1. Zero Moment Point (ZMP). The ZMP is where the ground reaction force acts whereas 
the CM point is where inertia and the force of gravity originate. 




Fig. 2. A legged posture is shown in which the ZMP is located just beneath the CM, the 
horizontal ground reaction force field is tangent to a circle centered about the ZMP, and the 
horizontal ground reaction force magnitude is a function of only radial distance. In this case, 
the net horizontal force is zero, but the net moment is nonzero. Thus, both the vertical 
component of moment and the CM work performed by the ground reaction force cannot be 
computed solely on the bases of the ZMP trajectory and the resulting ground reaction force 
vector. 



2.2. Centroidal Moment Pivot (CMP) 
2.3.1. Motivation 

Biomechanical investigations have determined that for normal, level-ground human 
walking, spin angular momentum, or the body's angular momentum about the CM, 
remains small through the gait cycle. Researchers discovered that spin angular momentum 
about all three spatial axes was highly regulated throughout the entire walking cycle, 
including both single and double support phases, by observing small moments about the 
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body's CM (Popovic, Gu and Herr 2002) and small spin angular momenta (Popovic, 
Hofmann, and Herr 2004a; Herr and Popovic 2004). In the latter investigations on spin 
angular momentum, a morphologically realistic human model and kinematic gait data were 
used to estimate spin angular momentum at self-selected walking speeds. Walking spin 
values were then normalized by dividing by body mass, total body height, and walking 
speed. The resulting dimensionless spin was surprisingly small. Throughout the gait cycle, 
none of the three spatial components ever exceeded 0.02 dimensionless units 2 . 
To determine the effect of the small, but non-zero angular momentum components on 
whole body angular excursions in human walking, the whole body angular velocity vector 
can be computed, or 

a>(t) = I- l (r CM ,t)L(r CM ,t) . (7a) 

N 

Here the time dependent quantity, l(r CM ,t) = ^I i (r CM ,t)f is the whole body inertia tensor 

i=\ 
about the CM. Subsequently, the whole body angular velocity vector may be integrated to 
give the whole body angular excursion vector, or 

t 
(9(0= \a(t*)dt*+C (7b) 

—00 

where C is an integration constant determined through an analysis of boundary conditions 3 
(Popovic, Hofmann, and Herr 2004a). The whole body angular excursion vector can be 
accurately viewed as the rotational analog of the CM position vector (i.e. note that 

analogously v = r CM = M~ l p and ^ C m(0= ycM^t )dt +D). In recent biomechanical 

investigations, angular excursion analyses for level ground human walking showed that the 
maximum whole body angular deviations within sagittal (<1°), coronal (<0.2°), and 
transverse (<2°) planes were negligibly small throughout the walking gait cycle (Popovic, 
Hofmann and Herr 2004a; Herr and Popovic 2005). These results support the hypothesis that 
spin angular momentum in human walking is highly regulated by the central nervous system (CNS) 
so as to keep whole body angular excursions at a minimum. 

According to Newton's laws of motion, a constant spin angular momentum requires 
that the moments about the CM sum to zero. During the flight phase of running or 



2 Using kinematic data from digitized films (Braune and Fisher 1895), Elftman (1939) 
estimated spin angular momentum during the single support phase of walking for one 
human test subject, and found that arm movements during walking decreased the rotation 
of the body about the vertical axis. Although Elftman did not discuss the overall magnitude 
of whole body angular momentum, he observed important body mechanisms for 
intersegment cancellations of angular momentum. 

3 Since the whole body angular excursion vector defined in equation (7b) necessitates a 
numerical integration of the body's angular velocity vector, its accurate estimate requires a 
small integration time span and a correspondingly small error in the angular velocity 
vector. 
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jumping, angular momentum is perfectly conserved since the dominant external force is 
gravity acting at the body's CM. However, during the stance period, angular 
momentum is not necessarily constant because the legs can exert forces on the ground 
tending to accelerate the system (Hinrichs, Cavanagh and Williams 1983; Raibert, 1986; 
Dapena and McDonald 1989; LeBlanc and Dapena 1996; Gu 2003). Hence, a legged 
control system must continually modulate moments about the CM to control spin 
angular momentum and whole body angular excursions. For example, the moment 
about the CM has to be continually adjusted throughout a walking gait cycle to keep 
spin angular momentum and whole body angular excursions from becoming 
appreciably large. To address spin angular momentum and the moment about the CM 
in connection with various postural balance strategies, the CMP ground reference point 
was recently introduced (Herr, Hofmann, and Popovic 2003; Hofmann 2003; Popovic, 
Hofmann, and Herr 2004a). Goswami and Kallem (2004) proposed the same point in an 
independent investigation 4 . 

2.3.2. Definition 

The Centroidal Moment Pivot (CMP) is defined as the point where a line parallel to the 
ground reaction force, passing through the CM, intersects with the external contact 
surface (see Figure 3). This condition can be expressed mathematically by requiring that 
the cross product of the CMP-CM position vector and the ground reaction force vector 
vanishes, or 

{r CM p ~ 7 cm ) x Fg.r. = and z CMjD = . (8) 

By expanding the cross product of equation (8), the CMP location can be written in terms of 
the CM location and the ground reaction force, or 

_ _ 1 G.R.X 
X CMP ~ X CM ~ ~ Z CM and ( 9 ) 

F G.R.Z 

F 

_ 1 G.R.Y 
y CMP ~ y CM ^ Z CM ' 
^ G.R.Z 

Finally, by combining ZMP equation (6) and CMP equation (9), the CMP location may also 
be expressed in terms of the ZMP location, the vertical ground reaction force, and the 
moment about the CM, or 

T y^c M ) J 

X CMP = X ZMP + — and ( 10 ) 

^ G.R.Z 



ycMP ~ yz 



F 

1 G.R.Z 



4 Popovic, Hofmann, and Herr (2004a) named the specified quantity the Zero Spin Center of 
Pressure (ZSCP) point, whereas Goswami and Kallem (2004) named the specified quantity 
the Zero Rate of Angular Momentum (ZRAM) point. In this manuscript, a more succinct 
name is used, or the Centroidal Moment Pivot (CMP) (Popovic, Goswami, and Herr 2005). 
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Fig. 3. Centroidal Moment Pivot (CMP). The CMP is the point where the ground reaction 
force would have to act to keep the horizontal component of the whole body angular 
momentum constant. When the moment about the CM is zero (shown in the figure to the 
right), the CMP coincides with the ZMP. However, when the CM moment is non-zero 
(figure on the left), the extent of separation between the CMP and ZMP is equal to the 
magnitude of the horizontal component of moment about the CM, divided by the normal 
component of the ground reaction force. 

As is shown by equation (10), when the CMP is equal to the ZMP, the ground reaction force 
passes directly through the CM of the body, satisfying a zero moment or rotational 
equilibrium condition. In distinction, when the CMP departs from the ZMP, there exists a 
non-zero body moment about the CM, causing variations in whole-body, spin angular 
momentum. While by definition the ZMP cannot leave the ground support base, the CMP 
can — but only in the presence of a significant moment about the CM. Hence, the notion of 
the CMP, applicable for both single and multi-leg ground support phases, is that it 
communicates information about whole body rotational dynamics when supplemented 
with the ZMP location (excluding body rotations about the vertical axis). 
It is interesting to note that when the stance foot is at rest during single support, and when 
there is zero moment about the CM, the ZMP and CMP coincide. However, generally 
speaking, these ground reference points cannot be considered equivalent. 



3. The ZMP, FRI and CMP trajectories in human walking 

For the diversity of biological motor tasks to be represented in a robot's movement 
capabilities, biomechanical movement strategies must first be identified, and legged control 
systems must exploit these strategies. To this end, we ask what are the characteristics of the 
ZMP and CMP ground reference points in human walking, and how do they interrelate? As 
discussed in Section 2.0, spin angular momentum remains small throughout the walking 
cycle. Hence, we hypothesize that the CMP trajectory will never leave the ground support 
base during the entire walking gait cycle, closely tracking the ZMP trajectory during both 
single and double support phases. 
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In this section we test the CMP hypothesis using a morphologically realistic human model 
and kinetic and kinematic gait data measured from ten human subjects walking at self- 
selected forward walking speeds. In Section 3.1, we outline the experimental methods used 
in the study, including a description of data collection methods, human model structure and 
the analysis procedures used to estimate, compare and characterize the reference point 
biological trajectories. Finally, in Section 3.2, we present the experimental results of the gait 
study, and in Section 3.3, we discuss their significance. 

3.1 Experimental Methods 

3.1.1 Kinetic and Kinematic Gait Measures 

For the human walking trials, kinetic and kinematic data were collected in the Gait 
Laboratory of Spaulding Rehabilitation Hospital, Harvard Medical School, Boston, MA, in a 
study approved by the Spaulding Committee on the Use of Humans as Experimental 
Subjects. Ten healthy adult participants, five male and five female, with an age range from 
20 to 38 years old, were involved in the study. 

Participants walked at a self -selected forward speed over a 10-meter long walkway. To 
ensure a consistent walking speed between experimental trials, participants were timed 
across the 10-meter walking distance. Walking trials with forward walking speeds within a 
±5% interval were accepted. Seven walking trials were collected for each participant. 
To assess gait kinematics, an eight infrared-camera, motion analysis system (VI CON 512 
System, Oxford Metrics, Oxford, England) was used to measure the three-dimensional 
positions of reflective markers placed on various parts of each participant's body. The frame 
rate of the camera system was 120 frames per second. A total of 33 markers were employed: 
sixteen lower extremity markers, five thoracic and pelvic markers, eight upper extremity 
markers, and four head markers. The markers were attached to the following bony 
landmarks: bilateral anterior superior iliac spines, posterior superior iliac spines, lateral 
femoral condyles, lateral malleoli, forefeet and heels. Additional markers were rigidly 
attached to wands over the mid-femur and mid-shaft of the tibia. Kinematic data of the 
upper body were also collected with markers placed on the following locations: sternum, 
clavicle, C7 vertebra, T10 vertebra, head, and bilaterally on the shoulder, elbow and wrist. 
Depending on the position and movement of a participant, the system was able to detect 
marker position with a precision of a few millimeters. 

During walking trials, ground reaction forces were measured synchronously with the kinematic 
data using two staggered force platforms (model OR6-5-1, AMTI, Newton, MA) embedded in 
the 10-meter walkway. The force data were collected at a sampling rate of 1080 Hz at an absolute 
precision of ~0.1 N for ground reaction forces and ~lmm for the ZMP location. 

3.1.2 Human Model Structure 

A morphologically realistic human model was constructed in order to calculate the FRI and 
CMP ground reference trajectories. The human model, shown in Figure 4, consisted of 18 
links: right and left forefoot links, heels, shanks, thighs, hands, forearms, upper arms, 
pelvis-abdomen region, thorax, neck and head. The forefoot and a heel sections, as well as 
the hands, were modeled as rectangular boxes. The shanks, thighs, forearms and upper 
arms were modeled as truncated cones. The pelvis-abdomen region and the thoracic link 
were modeled as elliptical slabs. The neck was modeled as a cylinder, and the head was 
modeled as a sphere. 
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Fig. 4. The morphologically realistic human model used in the human gait study. The 
human model has a total of 38 degrees of freedom, or 32 internal degrees of freedom (12 for 
the legs, 14 for the arms and 6 for the rest) and 6 external degrees of freedom (three body 
translations and three rotations). Using morphological data from the literature and direct 
human participant measurements, mass is distributed throughout the model's links in a 
realistic manner. 

To increase the accuracy of the human model, twenty-five length measurements were taken 
on each participant: 1) foot and hand length, width and thickness; 2) shanks, thighs, 
forearms and upper arm lengths as well as their proximal and distal base radii; 3) thorax 
and pelvis-abdomen heights, widths and thicknesses; and 4) radius of the head. The neck 
radius was set equal to half the head radius. 

Using observations of the human foot's articulated bone structure (Ankrah and Mills 2003), 
the mass of the forefoot was estimated to be 20% of the total foot mass. For the remaining 
model segments, a link's mass and density were optimized to closely match experimental 
values in the literature (Winter 1990; Tilley and Dreyfuss 1993) using the following 
procedure. The relative mass distribution throughout the model, described by a 16- 
component vector D , (i.e. the heel and forefoot were represented as a single foot segment) 
was modeled as a function of a single parameter a such that 



D(a) = (D A + aD s )/(l + a) . 



(11) 



Here D A is the average relative mass distribution obtained from the literature (Winter 1990), 
and the subject specific relative mass distribution, Z^ , was obtained by using an equal 

density assumption; the relative mass of the i-th link, D s l ,was assumed to be equal to the 

ratio of the link's volume, V 1 , over the total body volume, V , or D s l = V 1 jV . The selection 
of parameter a then uniquely defined the density profile throughout the various links of 
the human model, as described by the 16-component vector P(a) , such that 
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P l (a) = M D l {a)/V l where M was equal to the total body mass. The resulting relative 
mass distribution, D R , was obtained as D R = D(a min ) where a min minimized the absolute 
error between the distribution of link densities, P(a) , and the average distribution of link 
densities obtained from the literature, P^ (Winter 1990). In notation form, this analysis 
procedure may be expressed as 

, n D A+<Xwm D S 



i\P(a)-P A \ = min l^P* (a) - P$ 



(12) 



3.1.3 Data Analysis 

For each participant and for each walking trial, the ZMP and CMP trajectories were 
computed. The ZMP was estimated directly from the force platform data using equation (1). 
The CMP was calculated using the calculated CM position from the human model, and the 
measured ZMP and ground reaction force data from the force platforms (see equation (9)). 
Here the CM trajectory was estimated by computing the CM of the human model at each 
gait posture throughout the entire gait cycle. The model's posture, or spatial orientation, 
was determined from the joint position data collected from the human gait trials. 
As a measure of how well well the CMP tracked the ZMP, we computed the linear distance 
between the CMP and the ZMP, at each moment throughout the gait cycle. For each 
participant, the mean CMP-ZMP distance was then computed using all seven gait trials. 
This mean distance was then normalized by the participant's foot length. We then 
performed a nonparametric Wilcoxon signed rank test for zero median to test for 
significance in the mean CMP-ZMP distance between the single and double support phases 
of gait (N=10 subjects). For this statistical analysis, significance was determined using p < 
0.05. 

3.2. Results 

Representative trajectories of the ZMP and CMP are shown in Figure 5 for a healthy female 
participant (age 21, mass 50.1 kg, height 158 cm, speed ~1.3 m/s). For each study 
participant, Table 1 lists the mean normalized distances between the CMP and the ZMP. 
For all participants and for all walking trials, the ZMP was always well inside the ground 
support base. The ZMP was never closer to the edge of the ground support base than by 
approximately 5-10% of foot length (see Figure 5). Finally, for all participants and for all 
walking trials, the CMP remained within ground support base throughout the entire gait 
cycle. The mean of the normalized distance between the CMP and the ZMP for the single 
support phase (14 ± 2%) was not significantly different from that computed for the double 
support phase (13 ±2%) (p=0.35). 

3.3 Discussion 

Since spin angular momentum has been shown to remain small throughout the walking 
cycle, we hypothesize that the CMP will never leave the ground support base throughout 
the entire gait cycle, closely tracking the ZMP. The results of this investigation support this 
hypothesis. We find that the CMP never leaves the ground support base, and the mean 
separation distance between the CMP and ZMP is small (14% of foot length), highlighting 
how closely the human body regulates spin angular momentum in level ground walking. 
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The mean normalized distance between the CMP and the ZMP for the single support phase 
(14 ±2%) was not significantly different from that computed for the double support phase 
(13 ±2%) (p=0.35), suggesting that the CMP is a reasonable estimate of ZMP position 
independent of the number of legs in contact with the ground surface. 




& tUO 2DD 30U 

M#dlal -Lateral direction {mm} 



Fig. 5. Plotted are the ZMP (dashed), CMP (solid) and CM ground projection (dashed- 
dotted) trajectories and corresponding footprints of a study participant walking at a self- 
selected speed (1.3 m/s). The two circles on each line denote the transition from single to 
double support and vice versa. Data span from the middle of a single support phase to the 
middle of the next single support phase of the opposite limb. 
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Mean±STD 


A% 


16 


14 


13 


17 


16 


10 


12 


11 


15 


15 


14 ±2 


B% 


15 


13 


10 


15 


12 


9.0 


14 


15 


15 


14 


13 ±2 


C% 


16 


13 


12 


16 


15 


10 


12 


12 


15 


15 


14 ±2 



Table 1. For ten healthy test participants walking steadily at their self-selected speeds, listed 
are the mean distances, normalized by foot length, between the CMP and the ZMP points for 
the single support phase (A), double support phase (B), and across the entire gait cycle (C). 
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4. Control Implications of Ground Reference Points ZMP and CMP 

In this section, we discuss how the ZMP and CMP ground reference points can be used in 
legged robotic and prosthetic control systems. In Section 4.1, the control implications of the 
foot-ground ZMP are discussed. In Section 4.2, we discuss how the control of both the ZMP 
and the CMP could enhance postural stability for single-leg standing. 

4.1. Control Implications of the ZMP 

4.1.1 Does a ZMP Location Inside the Ground Support Base Indicate Postural 
Stability? 

As noted by Goswami (1999), the requirement that the ZMP should be inside the ground 
support base has been extensively used in the literature as a criterion of postural stability 5 
(Shih et al. 1990; Li, Takanishi and Kato 1993; Shih 1996; Arakawa and Fukuda 1997; Huang 
et al. 2001). However, since the ZMP must always reside within the ground support base as 
required by fundamental physics (see equation (1)), a ZMP estimate that falls outside the 
ground support base should be an indication of non-physical behavior and not an indication 
of overall postural instability. For example, if a computer simulation predicts that the ZMP 
is outside the ground support base, the result should simply be viewed as a non-physical 
simulation artifact and not an indication of postural instability. Still further, if the simulation 
predicts a ZMP location within the ground support base, overall postural stability is not, in 
any way, guaranteed. 

4.1.2 Does Maintaining the ZMP at the Center of the Ground Support Envelope 
Guarantee Postural Stability? 

It has been suggested in the literature that postural stability during single support will be 
ensured if the ZMP remains at the center of the ground support envelope (Vukobratovic 
and Juricic 1969; Vukobratovic and Stepanenko 1973; Li, Takanishi and Kato 1993; Arakawa 
and Fukuda 1997; Huang et al. 2001). However, it is noted here that accurately controlling 
the ZMP location to coincide with the center of the ground support envelope will not in 
itself guarantee postural stability for all legged control problems. To clarify this point, 
consider the simple model of single support standing shown in Figure 6A. The mass of the 
body is represented as a point mass attached to a massless foot and leg linkage, and the 
ankle is the only actuated degree of freedom. 
If the ZMP is tightly controlled to operate at the center of the ground support envelope, 

such that X ZMp = , then according to equation (6) 

F x = Mt CM =M{z CM + g )^M -^. (13) 

Z CM Z CM 

For this simplified model, the moment about the CM is always equal to zero, r y = , since 
the mass of the body is represented as a single point mass. Thus, from equation (13) we have 



5 Throughout this manuscript, postural stability, or body stability, refers to the maintenance 
of body attitude angles within a specified bounded region and the return to that bounded 
region after a perturbation (Vukobratovic, Frank and Juricic 1970). 
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*CM ~ \ Z CM + g) 



X CM 
Z CM 



(14) 



We see from equation (14) that for this simplified model, a control system that 
maintains the ZMP position at the center of the ground support envelope, or X ZMP = , 

causes the system to be equivalent to a statically unstable, non-actuated inverted 
pendulum. Thus, we may conclude that controlling the ZMP to operate at the center of 
the ground support envelope during single support cannot, by itself, ensure postural 
stability. 



A) 



heavy link 





ZMP 



X Z mp -0 



ZMP 



X ZMP ~ 



Fig. 6. In (A), a simple model of single-leg standing is shown consisting of three links: 1) a 
body link represented by a point mass equal to total body mass; 2) a massless leg link 
representing the stance leg; and 3) a massless foot link (base of support), which is aligned 
with the ground and which has limited extent. The ankle joint between the foot link and the 
leg link is the only actuated degree of freedom in the model. In (B), the same model as in (A) 
is shown except the body link is modeled as a solid uniform rod. In contrast to the model of 
(A), the model of (B) has an actuated ankle and hip joint. Thus, this model may have a non- 
zero moment about its CM. 



If we now allow for non-zero ZMP positions, we obtain 



*CM = {Z C M + g) 1 



CM 



(15) 



'CM 



Thus, we see from equation (15) that by selecting an appropriate non-zero ZMP trajectory, 
the model of Figure 6A can be stabilized albeit for relatively modest CM disturbances. 6 
For example, if the CM projection onto the ground extends beyond the boundaries of the 
foot as a result of a disturbance to the system, the system cannot be stabilized simply by 
controlling ZMP position because the foot is not physically attached to the ground surface 
(see equation (15)) (Popovic and Herr 2003; Hofmann et al. 2004, Popovic, Hofmann and 
Herr 2004b). 



6 Here stability refers to the capacity of the system to restore the CM to a location vertically 
above the center of the ground support envelope ( X ZMp = ) after a perturbation. 
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Although controlling ZMP position is one strategy for stabilizing legged posture, it is not 
the only tool for addressing stability. For example, during single-leg standing, consider 
shrinking the stance foot to a single point. The ZMP is then constrained at that contact point 
and cannot be repositioned using a ZMP control strategy. As is apparent from equation (13), 
the only way to stabilize such a system is to produce a non-zero moment about the CM. In 
Section 4.2, we argue that by controlling both the ZMP and CMP ground reference 
positions, overall postural stability during single support standing can be maintained even 
in the presence of large disturbances where the CM projection on the ground surface 
extends beyond the ground support envelope. 

4.2. Control Implications of the CMP 

4.2.1 For Whole Body Rotational Control, Should a Control System Minimize CM 

Moment, Spin Angular Momentum, or Whole-Body Angular Excursions? 

As noted in Section 3.2, the CMP trajectory was confined to the ground support base for 
each subject and for each walking trial. Thus, one metric of human-like walking that 
may be useful in the evaluation of biomimetic humanoid robots is that the CMP must 
remain within the ground support base, near the ZMP, throughout the entire gait cycle. 
However, a zero moment about the CM, or a zero CMP-ZMP separation, should only be 
viewed as a condition of body rotational equilibrium and not a condition of postural 
stability. A loss of rotational equilibrium does not necessarily mean that the person or 
robot is destined to fall. In fact, the moment about the CM is prominently non-zero for 
many stable legged movement patterns (Hinrichs, Cavanagh and Williams 1983; 
Dapena and McDonald 1989; LeBlanc and Dapena 1996; Gu 2003). Non-zero CM 
moments are expected since the application of CM moment by a legged control system 
can increase the restoring force applied to the CM, as shown by equation (6), restoring 
CM position to a desired location (Popovic, Hofmann, and Herr 2004a,b; Hofmann et al. 
2004). 

Since the application of moments about the CM is one critical control strategy to achieve 
postural stability in the presence of disturbances, the objective for the controller of whole- 
body angular behavior should not be to achieve a zero CM moment, or equivalently, a 
zero CMP-ZMP separation. Rather, a CM moment should be applied by the system 
controller to achieve a desired spin angular momentum and a particular whole-body 
angular excursion (see equation (7)). For example, focusing solely on rotational degrees of 
freedom, one could write a simple 2 nd order differential control equation for a desired 
target moment, or 

*des. (rcM ) = ^des. +aA6+bA6= L des +aA0 + b Al(F CM ) , (16) 



where AL{f CM ) = L{r CM )-L des i? CM ) and A0 = 0-0 des , a and b (with b =br l {f CM )) are second 
order tensors, i.e. 3x3 matrices representing rotational "stiffness" and "damping" 
coefficients respectively, j^ ) = V / /? x is the whole body moment of inertia tensor about 

the CM (also a function of time) and & = I~ l (r c )L(r c ) is the whole body effective angular 
velocity (that may be integrated to give 6 ), see equation (7). Alternatively, instead of whole 
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body angular excursions, which are not directly measurable quantities, one may consider 
using whole body principal angles defined by the relative orientations of the principal axes 
of the whole body moment of inertia tensor with respect to the non-rotating lab frame axes 
(Popovic and Herr 2005). For a humanoid walking robot, the desired whole body angular 
excursion and the spin angular momentum would both be set to zero and the rotational 
stiffness and damping coefficients would then be adjusted to achieve a desired system 
response. 

In his book Legged Robots that Balance, Raibert (1986) speculated that a control system that 
keeps angular momentum constant during stance could achieve higher efficiency and better 
performance. Motivated by biomechanical measurements showing the relatively small size 
of CM moments during human walking, Popovic, Gu and Herr (2002) suggested that 
humanoid control systems should explicitly minimize global spin angular momentum 
during steady state forward walking (L des (r CM ) = o)- Using this approach, the zero-spin 
controller would apply corrective moments to minimize body spin when the whole body 
state is such that spin is non-zero. It is noted here that a consequence of this control 
objective is that the CMP-ZMP separation distance is minimized. However, a control system 
that only minimizes the CMP-ZMP separation distance will only ensure a constant spin 
angular momentum and not specifically a zero spin value. 

Kajita et al. (2003; 2004) implemented a zero-spin control on the humanoid robot HRP-2 and 
showed its usefulness in kicking, hopping and running. Still further, Popovic, Hofmann and 
Herr (2004a) showed in a 2-D numerical simulation of walking that biologically realistic leg 
joint kinematics emerge through the minimization of spin angular momentum and the total 
sum of joint torque squared (minimal effort criteria), suggesting that both angular 
momentum and energetic factors may be important considerations for biomimetic 
controllers. 

4.2.2 Would Controlling Both ZMP and CMP Enhance Postural Stability? 

For the simplified model of single-leg standing shown in Figure 6A, Section 4.1, ankle torques 
have to be applied to move the ZMP such that appropriately needed horizontal forces are 
generated, as dictated by equation (15), to move the model's CM back over the foot support 
envelope. However, as required by physics (see equation (1)), the ZMP cannot leave the 
ground support base. This physical constraint poses a restriction on the magnitude of the 
restoring CM forces that can be applied by the system controller to restore CM position, and 
therefore, directly limits the range of perturbation that can be rejected by the system. 
Let us now relax the zero moment condition (CMP=ZMP) and consider the model shown in 
Figure 6B. In that model, the point mass of model 6 A is replaced with a uniform rod that 
rotates about a hip joint at the top of a massless leg and foot linkage. By controlling both the 
ZMP and CMP trajectories, a larger set of perturbations can be rejected than when controlling 
only the ZMP trajectory (Popovic, Hofmann and Herr 2004a,b; Hofmann et al. 2004). Even 
when the ZMP is at the very edge of the ground support envelope in the model of Figure 6B, a 
horizontal restoring force can still be produced through the application of a moment about the 
CM, or equivalently by controlling the CMP relative to the ZMP. According to equation (6), 
the horizontal restoring force output of the model shown in Figure 6B can now be written as 



71/fU A/f('^ i sS\ CM ZMP y 77 zero-moment . r?moment (1.7) 

Mx CM= M \ Z CM+g) = Ac + Ac K } 
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where p zero-moment _ j^(~ + \ x cm x zmp corresponds to a Zero-Moment Balance Strategy 

Z CM 
T 

and F x = corresponds to a Moment Balance Strategy. Because the CMP 

Z CM 

represents a unique pivot point, equation (17) may be written more compactly as 



F x =Mi(z CM+g Y CM *"" . (18) 

Z CM 

As highlighted by equation (18), the CM restoring force can be controlled by modulating the 
separation distance between the CM projection on the ground surface and the CMP location. 
Depending on the character of a particular movement task and robotic structure, the two balance 
control strategies may have different levels of influence on postural stability. For example, in 
Figure 6B, if the model's foot link were made infinitely small, with x ZMP = as a physical 

constraint, the Moment Balance Strategy (CMP ^ ZMP) would necessarily dominate. However, 
when the CMP is in the vicinity of the ground support envelope boundary during single-leg 
balancing, or outside that boundary, the Moment Balance Strategy (CMP ^ ZMP) must dominate 
since ZMP trajectory control alone cannot restore postural balance (Popovic, Hofmann and Herr 
2004a,b; Hofmann et al. 2004). Therefore, the CMP location relative to the ground support 
envelope is an important indicator for a control system to determine which balance strategy 
should necessarily dominate (Popovic, Hofmann and Herr 2004a,b; Hofmann et al. 2004). 

5. Summary 

For the diversity of biological motor tasks to be represented in a robot's movement 
repertoire, biomechanical movement strategies must first be identified, and legged robotic 
control systems must exploit these strategies. To this end, in this chapter we ask what are 
the characteristics of the ZMP and CMP ground reference trajectories in human walking, 
and how do they interrelate? We compute walking reference trajectories using a human 
model and gait data measured from ten human subjects walking at self-selected speeds. We 
find that the CMP never leaves the ground support base, and the mean separation distance 
between the CMP and the ZMP is small (14% of foot length) across both single and double 
support walking phases, highlighting how closely the human body regulates spin angular 
momentum in level ground walking. 

We conclude the chapter with a discussion of legged control issues related to the ground 
reference points. Using a simple model of single-leg balancing, we show that by 
controlling both the ZMP and the CMP trajectories, larger CM restoring forces can be 
applied by a system than would be possible using only a ZMP control. An area of future 
research of considerable importance will be in the implementation of legged systems that 
control both the ZMP and the CMP locations, resulting in corrective CM forces and 
moments necessary to restore CM position and body angular orientation. Another area of 
future research will be to characterize the ZMP and CMP biological trajectories for a 
whole host of animal and human movement patterns in the hope to further motivate 
biomimetic control schemes. It is our hope that this work will lead to further studies in 
ground reference points for the identification and control of legged systems, resulting in 
an even wider range of locomotory performance capabilities of legged robots and 
prostheses. 
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6. Appendix: Center of Pressure (CP) and Zero Moment Point (ZMP): 
Equivalence and Uniqueness 

Although several authors (Goswami 1999; Sardain and Bessonet 2004) have speculated that 
the CP should be equivalent to the ZMP 7 , no formal proof has yet been advanced. In this 
Appendix, we put forth a formal proof of their equivalence for horizontal ground surfaces, 
and then we show their uniqueness for more complex contact topologies. 

The Equivalence of the ZMP and the CP for Horizontal Contact Surfaces 

The concept of CP most likely originated from the field of fluid dynamics. CP is utilized 

in aero-dynamical calculations of aircraft and rockets (Darling 2002). It is also frequently 

used in the study of human gait and postural balance (Winter 1990; Rose and Gamble 

1994). 

For a body resting on a flat horizontal ground surface, the position of the CP, denoted by 

r CP , is defined as 



r cp =- 



\rp(r) da 

± _ ?G.R.( )\ horizontal £ / ( A -l) 



\p(r) 



da F GJLZ S 



where the integration is over the ground support base (gsb), da is an infinitesimal element 
of the support surface located at r , p (f) is the pressure at that location, F is the vertical 

component of the resulting ground reaction force, and g is the gravitational acceleration. 
The second equality in equation (A.l) follows from p(r)da = dF c R 7 and y . g = o . 

The resulting moment exerted from the ground on the body about the origin of the lab 
reference frame (assumed here to be on the ground) is 

r G . R (0) \ horizo „ la ,= J (r x dF GR )\ horizonlal =- | ?x * ]p{r)da = ^x \r p{r)da ■ ( A - 2 ) 

gsb gsb V <5 J o gsb 

For simplicity we assume a horizontal ground surface in equations (A.l) and (A.2). 
However, the results may easily be generalized to include inclined surfaces as well if vector 

- <y is exchanged for «j_ , the unit vector normal to the surface and pointing away from 

/ o 

the ground. In addition, F G R ? has to be exchanged with F G R , , the component of the 

ground reaction force normal to the surface, and f {horizontal nas *° ^ e exchanged for r L 

where f L -n^ = . For a more complicated surface geometry, for example when two robot 

legs are posed on two surfaces of different inclination, the unique embedding surface does 
not exist. Below, we resolve this issue by considering an embedding convex volume instead 
of an embedding flat surface. The flat surface approach was first proposed by Takanishi et al. 
(1990) and later used by Sardain and Bessonnet (2004). 



7 Goswami (1999) and Sardain & Bessonet (2004) did not prove the equivalence of the CP 
and the ZMP, but rather, they proved the equivalence of two definitions of the ZMP (see 
Section 2.1 for ZMP definitions, equations (1) and (2)). 
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Given the definition of the CP (equation (Al)), we can prove that the CP is identical to the 
ZMP by noting from equation (A.2) that 



T G.R.\ r CP) I horizontal - T G.R. W I horizontal + ^ ' G.R.Z T CP X ~~ "' 

g 

therefore satisfying one definition of the ZMP defined in equation (1), Section 2.1. 
Alternatively one could rewrite equation (1), Section 2.1 as, 



T G.R. VZMP ) I horizontal ~ J |.V T ZMP ) X ^ G.R. {horizontal ~ J 



(r-f ZMP )*- 



dF n 



0' 



(A3) 



(A.4) 



to show that it is exactly satisfied when the ZMP is identical to the CP (equation (A.l)), or 

/ -\ J' G.R.Z 

To., (hMP ) L„- = " j [r x - kxz + 4— x " J>«.z =0 QED' {A3) 

gsb\ &J jdr GRZ & gsb 

gsb 

Hence, for a flat horizontal support base, the ZMP and the CP exactly coincide. 

The Uniqueness of the ZMP and the CP for Complex Contact Topologies 

Consider the human model shown in Figure 7. Here the model's hand and foot are exerting 
forces against a non-horizontal contact surface. Given the net CM force, the ground reaction 
force may be obtained by simply subtracting the gravitational force. Given the CM location 
and the net moment about the CM, the ZMP line may be constructed. The intersection of 
that line with the contact surface then defines the ZMP location. In distinction, the CP may 
be obtained by integrating across the contact surface according to the first equality of 
equation (A.l). Hence, the CP can be positioned anywhere inside the convex hull 
represented by a 3-D, CP embedding volume and encompassing the contact foot and contact 
hand. For this particular example, the CP is not a ground reference point at all but is located 
above the contact surface. 



ZMP line *^Z^r \ 




ZMP 

Fig. 7. Dynamical multi-link humanoid model with hand and foot contact. The ground 
reaction force originates at the ZMP. Inertia and the force of gravity originate at the CM 
point. As is shown in the figure, the ZMP and the CP point do not coincide for non- 
horizontal contact surfaces. 
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Using mathematical notation, we show that the ZMP is not equal to the CP for non- 
horizontal contact surfaces like those depicted in Figure 8. For a general distribution of a 

normal unit vector field, Wj_(r) ^ COflSt. , defined on all contact surfaces, one may show 
that the ZMP is not equal to the CP by first defining the ZMP as 



\{r-r CM )*dF R 



(f 



r)x \dF R 



(A.6) 



where T zup is on the external contact surface. One can then set this definition of the ZMP 



equal to the CP (equation (A.l)), to observe that 

jr(dF R -n) 



j(rxdF R \ or 



JdF~n) 



K. 



(A.7) 



Hence, the ZMP and the CP do not always coincide and should therefore not be considered 
identical physical quantities. It should be noted that for expressions (A.6) and (A.7) we 
avoided the prefix ground to stress that any type of external contact surface is permissible 
when the ZMP is defined according to equation (A.6). Also, using this formalization, any 
body segment may be in contact with the external surface. 
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Symbol Table (in alphabetical order) 

a. Body segment i center of mass acceleration. 

a Parameter used for optimization of human model mass 

parameters. 
£) The relative mass distribution described by a 16- 

component vector. 
D A The average relative mass distribution (Winter 1990). 

D R The resulting relative mass distribution. 

D s The subject specific relative mass distribution obtained 

by equal density assumption. 
The relative mass of the i-th link. 



F 



ankle 



Net force acting on a whole body (in free fall 
F=F= ^ F z =-Mg). 

The net force at the stance foot ankle joint exerted from 
the rest of the body. 
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G.R. 



G.R.. 






R. 
moment 



zero-moment 



g 

g 
I 
Ufa) 

^des. VCM ) 
M 

m i 

n ± 

Pa 
P(a) 

P i (a) = MD i (a)/v i 
p(r) 

r CM 

r CMP 



' ZMP 



f (r 



ZMP ) I horizontal 



)l 

T des. VCM ) 
^G.R.yy) I horizontal 
T G.R. VZMP ) I horizontal 



Ground reaction force. 

The component of the ground reaction force normal to 

the surface. 

Reaction force (general surface). 

Net zero force in x direction corresponding to the 
Moment Balance Strategy. 

Net zero force in x direction corresponding to the Zero- 
Moment Balance Strategy. 

Gravitational constant (9.%\ m / 2 ). 

/ s 
Gravitational vector ( - g • e z )• 

Body segment i inertia tensor about the link's center of mass. 

The time dependent segment / moment of inertia tensor 

about the center of mass. 

The time dependent whole body moment of inertia 

tensor about the center of mass. 

The desired whole body angular momentum. 

Body mass. 

Body segment / mass. 

The unit vector normal to the surface and pointing away 

from the ground. 

The average distribution of link densities (Winter 1990). 

Density profile described by the 16-component vector. 

Density of the i-th link. 

The pressure at location r . 
Body center of mass. 

Centroidal Moment Pivot point. 

Body segment / center of mass. 

Zero Moment Point. 

Time. 

Horizontal component (orthogonal to gravity vector) of 

the net moment about the Zero Moment Point. 

The desired target whole body moment about the center 

of mass. 

The resulting moment exerted from the ground on the 

body about the origin of the lab reference frame. 

Horizontal component (orthogonal to gravity vector) of the 

moment of ground reaction force about the Zero Moment Point. 
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Z (Z \ I Horizontal component (orthogonal to gravity vector) of 

inertia + gravity V ZMP ) \ horizontal , 7 , j , • ,. i j •..• ir i a. 

6 ' the moment due to mertial and gravitational torces about 

the Zero Moment Point. 
f In The component of the whole body moment parallel to the 

flat surface (i.e. f L -«_|_ = ). 

n The time dependent whole body angular excursion 

vector. 
n The desired target whole body angular excursion. 

U des. 



Odes. 



The desired target whole body angular acceleration. 

y The time dependent whole body center of mass velocity. 

(error in manuscript) 
y i The volume of the i-th link. 

(q The time dependent whole body angular velocity vector. 

q). Body segment i angular velocity. 

~ Body center of mass acceleration in the vertical direction 



(in free fall Z CM =-g] 
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1. Introduction 

Reproducing human dexterity and flexibility in unknown environments is one of the major 
challenges of robotics. Among the issues related to the use of robots with artificial hands of 
varying complexity, the definition of their kinematical configuration when grasping an object is 
one of the most difficult problems. Indeed, it necessitates the consideration of a large number of 
constraints related to the structure of the hand as well as the object, the requirements of the task 
and the state of the environment. In this frame, sensing capability is crucial to obtain information 
about the robot state and the quality of sensory signals that may be noisy or of low precision has to 
be considered to control efficiently a robotic grasping task. 

In the literature, several paradigms have been proposed to perform grasp planning. Mainly, 
they can be divided in three categories: knowledge based grasp planning, geometric grasp 
planning and sensory driven or learning based grasp planning. 

Knowledge based grasp planning refers to techniques where posture definition is made 
according to direct or indirect knowledge in human prehension. In fact, in order to better 
understand the different parameters involved during grasping, several studies have been 
conducted to classify human prehension. Their main principle is that prototypical hand shapes 
are used to perform classes of tasks according to particular hand, environment and task related 
constraints (Napier, 1956; Iberall et al., 1986; Cutkosky, 1987; Tomovic et al., 1987; Bekey et al, 
1993; Iberall, 1997; Saito & Nagata, 1999). These classifications have been intensively used to 
propose solutions in the robotics field, especially in the frame of artificial intelligence and expert 
systems (Pook & Ballard, 1993; Kang & Ikeuchi, 1997;). The concept of " assembly plan from 
observation 77 states that a robot can perform complicated grasps and manipulations by taking 
advantage of the human dexterity. Thus, optimal solutions can be obtained by the direct 
observation of human behavior and the extraction of the appropriate movement features 
(Jeannerod, 1984; Bard et al., 1991; Lee & Xu, 1996; Becker et al, 1999). These methods are very 
effective but they need sophisticated motion sensing devices such as datagloves. Moreover, the 
robot can learn only what is demonstrated and it is difficult to generalize from one task to 
another. Also, an alternative way to define hand posture is to use the concept of preshaping 
coming from experimental studies on prehension. In fact, it has been demonstrated (Wren & 
Fischer, 1995) that, prior to grasping, there is an anticipation of hand opening and orientation 
which is a function of object and task constraints. Therefore, the grasp planning problem is 
dramatically simplified by the choice of a finite number of hand preshapes. Then, hand closure 
schemes are defined in order to achieve grasp from the adequate preshape (Miller and al, 2003). 
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Geometric contact level grasp synthesis algorithms have been developed to find an 
optimal set of contacts according to grasp quality measures like force and form closure 
(Kerr & Roth, 1986; Nguyen, 1986; Ferrari & Canny, 1992; Gorce & Fontaine, 1994; Mirtich 
& Canny, 1994, Miller & Allen, 1999). In this case, the hand is represented by a set of 
contacts and while these algorithms are applicable to any dexterous hand they leave the 
problem of finding a suitable hand configuration open. Optimization based methods are 
then proposed to define the hand kinematical configuration (Borst et al., 2002; Guan & 
Zhang, 2003). The main drawback of such approaches is that they often rely on a precise 
knowledge of the task to be performed. 

Sensory driven grasp planning tries to solve this difficulty by incrementally constructing a 
model of the task by exploration and learning (Coehlo & Grupen, 1997; Wheeler et al., 2002; 
Grupen & Coehlo, 2002; Pelossof and al., 2004). In this frame, neural network based 
approaches are used to learn the underlying rules adopted during grasping. For example, 
(Kurper stein & Rubinstein, 1989) developed a neural network based scheme called INFANT 
in order to perform grasping tasks with a 5 degrees of freedom gripper. (Uno et al, 1995) 
proposed a neural based computational scheme able to integrate an internal representation 
of objects as well as the corresponding hand shape. (Taha et al., 1997) developed a model of 
preshaping of a planar hand model (with seven degrees of freedom) for circular and 
rectangular objects. (Moussa & Kamel, 1998) proposed a computational architecture able to 
learn grasping rules called " generic grasping functions". This representation treats grasping 
knowledge as a mapping between two reference frames: the object body attached coordinate 
frame, and the gripper body attached frame. This paradigm is applied to a very simple 
mechanism (single jaw gripper) and can not be applied to a multifingered hand where the 
mapping between the hand and the object frames may not be sufficient. 
In this chapter, a new method based on neural networks is proposed. It allows learning 
multichain redundant structures configuration during grasping in the presence of noise and 
uncertainty as well as unknown obstacles. Theses structures include serial manipulators 
equipped with a single jaw gripper or a multifingered hand. The developed model is composed 
of one generic module whose inputs and outputs vary according to the considered robot. A 
second module carries out the learning of the fingers inverse kinematics when a multifingered 
hand is used. It is based on a modular architecture composed of several neural networks. Using 
reinforcement learning, the main neural network based module optimizes the position and 
orientation of the grasping device taking into account noisy sensing information. Working 
together these two modules exchange information to define the complete robot configuration. 
The main interest of this architecture is that it is generic and can define the posture of hand, 
arm+hand or non anthropomorphic robots (e.g. the MANUS: a robotic assistant for the disabled) 
with noise or obstacles with few modifications. In order to illustrate the capabilities of the 
proposed model, simulation results are proposed using different situations considering different 
robot kinematical structures. 

This chapter is organized as following. In section 2, we describe the model architecture. The 
retained scheme to learn the inverse kinematics of the fingers is described in section 3. Then, 
section 4 is focused on the procedure used to define and optimize robot configuration. 
Finally in the remaining sections, simulation results are presented to demonstrate the 
efficiency of the proposed tools with considering the anthropomorphic hand only (section 
5), the hand + arm model (section 6) and a non anthropomorphic robot (section 7). 
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2. Model presentation 

The adopted approach will be explained considering the hand palm configuration only (i.e. 
position and orientation). The resolution process is almost the same when applied to the 
arm + hand model with only few modifications. 

Hand palm 





Object 



b/ 



Fig. 1. a/ 3D model of the hand, 



b/ Problem statement 



2.1 Hand model 

The hand model is composed of five articulated rigid chains representing the fingers. They 
are connected to a common body representing the palm (Fig. la/). Each finger has three 
links connected by three joints with a total of four degrees of freedom. The first joint of each 
finger has two degrees of freedom that allow to simulate the flexion-extension and 
abduction-adduction movements. The two other joints have one degree of freedom 
representing joint flexion. The complete model has 26 degrees of freedom. They are divided 
in two groups: 20 for the fingers joint angles and 6 for the hand palm position and 
orientation relative to a global frame. This architecture is generic and can be associated with 
many robotic hand designs directly as is or with simplifications with fewer fingers or fewer 
degrees of freedom. In the remainder of this article, the terms hand posture or grasping 
posture are used to define the numeric value of the set of parameters describing the 
kinematical configuration of the hand model during grasping. 



2.2 Problem definition and chosen hypotheses 

The definition of the problem is: Given an object and a contact set, define all the kinematics 
parameters of the hand model in such a way that all the fingertips can reach a defined contact position 
on the surface of the object to be grasped. This task has to be executed in the presence of noise and 
uncertainty. The number of contacts is between two and five (Fig. lb/). 
To define the hand posture, the following assumptions are made: 

1. Only precision grasps are considered, therefore there is only one contact per finger, 

2. We assume that the contact set is already defined and satisfies force closure, 

3. Each contact on the surface of the object is associated with a particular finger, 
The input data of the model are: 

1. The hand geometry, 

2. The number of contacts, 

3. The finger associated to each contact, 
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4. The location of the contacts in the object reference frame, 

5. The bounds of a 6D hand palm search space. 
The outputs of the model are: 

1. The location and orientation of the hand palm reference frame expressed in the 
world frame, 

2. The configuration of the fingers in joint coordinate space. 

2.3 Retained approach 

Defining all these kinematics parameters in a single step is a complex task because of the large 
number of degrees of freedom. One can consider it as an inverse kinematics problem of a 
redundant multichain mechanism with multiple objectives. Indeed, several difficulties arise: 
several fingers are involved and a hand palm configuration may conduct to a good solution for 
one finger but bad for the others. The second point is relative to the fact that an infinite number of 
solutions may exist. To solve this problem, we start from the idea that if the hand palm 
configuration is held fixed it is possible to compute the configuration of the fingers in joint 
coordinate space by using inverse kinematics. Furthermore, if we express the desired fingertip 
position in the frame of the finger and construct a model of inverse kinematics by learning, it is 
possible to compute quickly the fingers joint angles given any hand palm configuration. Thus, the 
number of unknown parameters decreases from to twenty six to six. These latter correspond to the 
hand palm configuration which remains to be defined. The chosen mechanism has to optimize the 
hand palm configuration in such a way that the fingers configuration allows the fingertip to reach 
the contact on the surface of the object. In order to integrate sensor noise and uncertainty, a 
stochastic search procedure based on reinforcement learning is chosen. The process of hand 
grasping posture definition has two parts: the hand palm configuration is generated by a first 
network (called "Hand Configuration Neural Network" or HCNN) and the corresponding finger 
joints are obtained by inverse kinematics with a second neural network model called "Finger 
Configuration Neural Network" or FCNN. Then the whole grasping posture is evaluated and 
from the corresponding evaluation the HCNN is trained by reinforcement learning in order to 
generate a better hand posture at the next iteration. 




Fig. 2. Model architecture. 



2.4 Model structure and principle 

According to the method presented in the former section, the model global structure is 
composed of two neural modules working in a closed loop (Fig. 2). The first module 
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(HCNN) is aimed at determining the appropriate hand palm configuration (location and 
orientation) relative to a global frame. It is a multilayer feedforward neural network with a 
learning process based on the paradigm of reinforcement learning. This type of learning is 
achieved with resorting to a particular type of processing unit called Stochastic Real Valued 
(SRV) neurons. The detailed structure of the HCNN is presented in section 4. The FCNN, 
the second module, is devoted to the definition of the fingers configuration in joint 
coordinate space from the desired fingertips position. Its output data are used to evaluate 
the appropriateness of the hand configuration by an evaluative function. The corresponding 
reinforcement signal is used by the HCNN to update its internal parameters. In section 3, 
the FCNN is presented in details. The principle of the developed method is now described: 

1. The current hand configuration and the difference between the actual and previous 
hand configurations (equivalent to a speed with unit time) are input to the HCNN. 
As output, a new hand palm configuration is obtained, 

2. Thanks to this new hand palm configuration, it is possible to express the position of 
the contacts in the frame of each corresponding finger (on the surface of the object, a 
particular contact is affected to each finger), 

3. Using the inverse kinematics scheme (FCNN), the joint configuration of the fingers is 
computed. 

4. The complete hand configuration (hand palm configuration and fingers joint angles) 
is tested with a criterion. From this latter, a reinforcement signal is computed and is 
used to update the HCNN internal parameters. Before this last step and in order to 
introduce noise and uncertainty, the actual reinforcement is perturbed with noise 
with zero mean and predefined variance, 

5. This procedure is repeated until a good solution is found (i.e. the reinforcement 
reaches a sufficiently high value and the criterion is optimized) or until the 
maximum number of iterations is reached 

The evaluative function evaluated after each iteration is based on the computation of the sum of 
the distance between each fingertip desired and actual position. In step 2, the location of the 
contacts is expressed in the frame of the fingers and therefore the desired fingertip position is 
available. For each finger, forward kinematics is used to evaluate the actual fingertip position. Let 



p X f4f,,/vf a) 

be the vector of the desired fingertip position expressed relative to the coordinate frame of 
finger i at step k and 

PX? ={ X ? ,y? ,z?J (2) 

be the vector of the actual fingertip position expressed relative to the coordinate frame of 
finger i. If n fingers are involved, the total error obtained at step k is: 



= X| PX ^" PX ^ 



(3) 



In order to model the effect of low quality sensors, this evaluative function is perturbed with 
a noise of known properties. The chosen procedure is detailed in section 4. The purpose of 
the next two sections is to provide further explanations on the structure and function of the 
two neural network based modules FCNN and HCNN. 
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3. Finger configuration neural networks (FCNN) 

Given a contact set in the reference frame of the object and a global hand palm 
configuration, the problem is to compute the configuration of the fingers in joint coordinate 
space. To carry out this task (Rezzoug and Gorce, 2001; Gorce and Rezzoug, 2005), a slightly 
modified version of the modular architecture defined by Oyama and Tachi (Oyama & Tachi, 
1999; Oyama & Tachi, 2000) is used. The main advantage of this method over other 
connectionist models is that it takes into account the discontinuity of inverse kinematics 
mapping of multi-joints mechanisms with joint limits by its modular nature. Indeed, it is 
difficult to approximate such a function by a single multi-layers neural network. The 
principle of the formalism defined by Oyama and Tachi (Oyama & Tachi, 1999; Oyama & 
Tachi, 2000) is based on three particular points. Firstly, the learning of inverse kinematics is 
performed by the incremental creation of multiple neural networks called experts each one 
spanning correctly one sub-set of the finger workspace. Secondly, when an expert is created, 
a representative posture in joint coordinate space is attached to it. Thirdly, after learning and 
for a given fingertip desired position, the finger joint angles are obtained by the appropriate 
competition among the created experts. 

An expert is a feedforward neural network with four layers. The activation function of the 
input and output layers is linear and tangent sigmoid for the hidden layers. The input layer 
is composed of three units that correspond to the X, Y and Z position of the fingertip. The 
output layer has 4 units corresponding to the 4 degrees of freedom of a finger. The two 
hidden layers have 35 units each. The error backpropagation algorithm is used to update the 
neurons weights. During learning, four to six experts are generated for each finger. 
In the remainder of this section, some details on the learning procedure are provided. 

3.1 Learning procedure 

At the beginning of the learning process, there is only one expert, with the representative 
posture [0, 0, 0, 0] T in joint coordinate space. For one particular finger, during one epoch of 
learning the following steps are performed: 

1. A fingertip desired position Xd in an appropriate 3D workspace is chosen randomly. 
Each expert produces an output representing the finger joint angles □,-. For each 
expert, the finger forward kinematics is used to compute the current fingertip 
Cartesian position Xc from the joint angles □;. The output error is the Euclidean 
distance between the current and the desired fingertip position (DDXc-XdDD). The 
posture generated by the expert with the minimal error is chosen as a start point. 
Then, a reaching motion represented by a straight line is constructed in Cartesian 
and joint spaces from the current to the desired fingertip position. The number of 
points that defines this motion is a function of the produced error. If the error is 
large, several points are stored. On the other hand, if the error is below a predefined 
threshold, an iterative procedure is used to improve the accuracy of the finger 
posture for the current training point (see section 3.2 for further details). Finally, the 
points of the reaching motion or the improved fingertip position and the 
corresponding joint angles are stored in the training set of the expert that has 
produced the minimal error among all the experts. Then, for a predefined number of 
steps, the learning of the training set is performed for the chosen expert. 

2. If during the reaching motion construction, no satisfactory solution is obtained for the best 
expert either due to joint limits or if the finger reaches a singular position, another expert is 
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selected in increasing order of the predicted error. The procedure described in step 1 is 
repeated until a satisfactory solution is found or until all the experts are tested, 

3. If no satisfactory solution is obtained during steps 1 and 2, an expert is randomly 
chosen and a reaching motion is constructed from the fingertip position derived from 
the expert representative posture to the desired fingertip position. This procedure is 
repeated until a solution is found or all the experts are tested, 

4. If no solution is obtained during steps 1, 2 and 3, a posture is randomly computed 
and a reaching motion is performed between the actual and desired fingertip 
position. This procedure is repeated until a satisfactory motion is generated (i.e. no 
singular position and joints within their limits). If so, the finger configuration 
obtained at the end of the reaching motion is considered as the representative 
posture of a new expert, which is added to the set of experts. 

3.2 Reaching motion and iterative improvement of fingertip position 

In this section, the reaching motion definition and the iterative improvement of fingertip 
position (Oyama & Tachi, 1999; Oyama & Tachi, 2000) are described. 

We consider 0(0), the output of one selected expert. Then, the error between the current and 
the desired fingertip position is computed: 

e(0)=\X n -g(Q(0))\ (4) 

X D represents the desired fingertip position and g(.) the finger forward kinematics. 
Two cases are considered: 

1. if e(0) < r s t , r $ t being a predefined threshold, perform the iterative procedure is 
performed and the joint angles Q(k+1) are computed according to the following 
equation: 

e(^+i) = e(^)+j + (e(^))-(x D -g(e(^)))untii^<r e (5) 

r e is a second predefined threshold. 

2. if e(0) > r $ t, a reaching motion is performed. To do this, a straight line is constructed 
in Cartesian space from Xc = g(0(0)) to Xd according to the following precdure: 

Let T be an integer satisfying the following inequality: 

|X n -X r | (f,) 

r-i<^ — ^<r ^ 

r st 
The Cartesian trajectory X D (/c) is constructed in the following way: 

, x [fl--l-X c +-.X D (0<k<T) (7) 

X D (*) = -R T) c T D V ; 

1 X D (k>T) 

If r s t is sufficiently low, the set of joint angles Q(k) that produces the trajectory of the fingertip 
as defined by (7) can be calculated in the following way: 

eO+i) = e(^)+r(e(^))-(x D (^+i)-g(e(^))) ( 8 ) 

J + represents the pseudo inverse of the finger Jacobian matrix. 
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During the iterative procedure, a single value of Q(k) is computed while several are 
determined during the reaching motion construction. This is due to the fact that, in the first 
case the error is considered small enough and it is just necessary to improve this particular 
output of the expert. Practically, at the beginning of the training phase, large reaching 
motions are constructed denoting the fact that the experts fail to approximate correctly the 
desired finger joint configuration. As training progresses, smaller reaching motions are 
generated and finally only the iterative procedure is performed. 

The function g(.) representing finger forward kinematics can be obtained from a previously 
trained neural network or directly from its closed form expression. For simplicity the latter 
procedure was chosen. The main difference between the architecture of (Oyama & Tachi, 
1999; Oyama & Tachi, 2000) and our implementation (Rezzoug & Gorce, 2001; Gorce and 
Rezzoug, 2005) is that while forward and inverse kinematics are learned by two separate 
modules, the closed form expression of forward kinematic is used and therefore only the 
inverse model is learned. At each iteration, the current generated motion only is learned by 
the best expert(Oyama & Tachi, 1999; Oyama & Tachi, 2000). On the other hand (Rezzoug & 
Gorce, 2001; Gorce and Rezzoug, 2005), we keep track of all the previous learned motions to 
speed up the learning. Also, every five epochs, all the training sets are tested by all the 
experts. If it happens that a point is best approximated by an expert, this point is transferred 
to the training set of the corresponding expert. Therefore, as training proceeds, we insure 
that the points of an expert training set are best learned by the corresponding expert. 

4. Hand configuration neural network (HCNN) 

4.1 Reinforcement learning with a direct method 

The learning process used to define the hand palm configuration (HCNN) is based on 
reinforcement learning (Sutton, 1988; Watkins, 1989; Kaelbling et al., 1996; Sutton & Barto, 1998; 
Doya; 1999). In this frame, a learning agent has to infer appropriate actions to perform a task based 
solely on evaluative feedback. Unlike supervised learning, the agent does not know what action to 
take in a given situation but has only limited information on how well it behaves after the 
execution of the action. This information called reinforcement can take the form of a binary signal 
such as (failure) or 1 (success) or be a real value. An extensive amount of relevant work has been 
proposed describing reinforcement learning problems (Sutton, 1988; Watkins, 1989; Kaelbling et 
al., 1996; Sutton & Barto, 1998; Doya, 1999). Most of the developed methods deals with discrete 
input and action spaces such as temporal difference learning (TD) (Sutton, 1988) or Q-learning 
(Watkins, 1989). The necessity to work in continuous input and action spaces, (e. g. to control 
robots) has led to the development of new methods based on an adequate discretization of state 
and action spaces or on the adaptation of the TD(D) algorithm (Doya; 1999). 

Another point of interest of this technique is the choice of the process to infer the appropriate 
actions from evaluations. Gullapalli (Gullapalli, 1990; Gullapalli, 1995) presented the two major 
categories, indirect and direct methods. Indirect methods rely on the model construction of the 
transformation from the controller's action to the evaluations. Then, this model is used to obtain 
gradient information for training the controller (Gullapalli, 1990). On the other hand, direct 
methods perform this task directly by perturbing the process. From the produced effects on the 
performance evaluation the agent is able to modify its internal parameters in order to maximize 
the reinforcement. Usually, the perturbation takes the form of a random noise with known 
properties and a stochastic search procedure is conducted (Gullapalli, 1990). This latter procedure 
seems to be very attractive, since no model of the process is necessary. Indeed, building such a 
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model is very difficult especially in the presence of noise and uncertainty. In the frame of direct 
methods, the process itself provides the necessary training data. That is the reason why we have 
chosen this formalism to determine the hand palm configuration by the mean of a neural network 
composed of backpropagation and Stochastic Real Valued (SRV) units which are detailed in 
section 4.2 The main interest of SRV units is that they enable the learning of functions with 
continuous outputs using a connectionist network with a direct method (Gullapalli, 1995). 
The neural network has 2 hidden backpropagation layers and an output layer composed of SRV 
units. The input layer has 12 units: 6 for the hand palm attached coordinate frame configuration 
and 6 for the difference between the actual and previous hand palm configuration. The 2 hidden 
layers have 20 units each. The association of SRV and backpropagation units allows to take 
advantage of both supervised and reinforcement learning. The whole network still has the 
benefits of reinforcement learning thanks to its stochastic search behavior. Also, the 
backpropagation units in the hidden layer enable to develop the right internal distributed 
representation of the problem as it is seen in supervised learning. 

In order to better understand how the SRV units learn, a description of their input-output 
relation and stochastic learning behavior is now presented. 

4.2 SRV unit input-output relation and learning process 

An input vector ik from X cz $R n , where $i is the set of real numbers, is presented to a SRV unit at 
time step k. The unit produces a random output Ok selected from some internal probability 
distribution over the interval O cz 91 The SRV unit uses its input ik to compute the parameters /& 
and ok of an internal normal probability distribution (jUk is the mean and ok the standard 
deviation). These parameters are obtained by the weighted sum of the input iy, with a particular 
set of weights for each parameter. We define the weight vectors (or matrices for a layer of SRV 
units) and & respectively for juy and ok. In the case of several SRV units <% and cptj corresponds 
to the weight associated with the j th component of the input vector and I th SRV unit parameter. 
For a single unit, the mean of the normal probability distribution is obtained as following: 

Hk=0 T h (9) 

6 is a column weight vectors. 

The computation of the standard deviation is done in two stages. Firstly, an expected 

reinforcement is computed as the weighted sum of the input vector k by the column vector qr. 

r k =<p T i k (10) 

Then, the standard deviation is obtained as a function of the expected reinforcement as: 

where s(.) is a monotonically decreasing, non negative function of r k . Moreover, s(1.0) = 0, 

so that when the maximum reinforcement is expected, the standard deviation decreases to 
zero 0. The expected reinforcement represents the internal estimation of the reinforcement 
obtained after taking a particular action. The standard deviation represents the degree of 
exploration around the mean. Therefore, if the expected reinforcement is high it is likely that 
the amount of exploration is low and therefore the standard deviation should be low. 
Once juk and ay are computed, the SRV unit computes its activation drawn form the normal 
distribution defined by juy and ay. 
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a k 



~N( Mk ,a k ) (12) 



Finally, the output is obtained as a function of the unit's activation as Ok =f(ajj. In the present case the 
chosen function/^.) is the logistic function, because the output is restricted to lie in the interval [0, 1]. 
In order to obtain an output vector within the desired bounds, the network output vector o\ 
is scaled according to the following equation: 

X z+1 = X min + ( X max ~ X min )® O k (13) 

Xi+i denotes the new output, Xmm the lower bounds of the search space, X max the upper bounds of 
the search space, Ok the network output vector and <8) the componentwise vector product. 
The environment evaluates the new output Xj+i according to the evaluation function (1-3) and the 
context of i\ and returns a reinforcement signal ^gR= [0, 1], with r\ = 1 denoting the maximum 
possible reinforcement. Therefore, the reinforcement signal value is obtained as follows: 

r k =l-h(E k ) (14) 

where E\ (3) corresponds to the error at time step k. h is a monotonic increasing function of the 
error Ek taking values over the interval [0, 1]. If Eu is large, h tends towards 1 and therefore the 
network receives a maximum punishment with a reinforcement toward 0. On the contrary, if the 
error Eu is low, h tends towards and, consequently, the system receives a higher reinforcement 
through equation (14). In the present case, h is the tangent sigmoid function. 

In order to model low sensing quality and noise effect, the actual reinforcement is perturbed 
with a random noise with a zero mean and known standard deviation a v (15). Also, it is 
considered to be distributed according to a Gaussian probability distribution. This random 
parameter reflects the quality of hand position sensors providing information about the 
hand palm configuration as well as fingers joint position sensors: 

r k =r k +N(0,a n ) (15) 

To update the two parameters 6(k) and <p(k) used in the computation of the mean juk and 
standard deviation c% the following learning rules are used: 



0(k + l)-- 



(^)P^ (0 if^>0 (16) 



6(k) + a 

6(k) if cr^ = 

cp(k + \) = cp(k) + p(r k -r k )i k (17) 



a and p are constant learning rates. 

The update rules are designed to produce the following behavior. If the normalized perturbation 

added to the mean output of the unit (fraction in (16)) leads to a reward that is greater than the 

expected reward, then it is likely that the unit produces an output that is closer to the actual 

output. In other words, the mean should be changed in the direction of the perturbation that has 

produced a better reward and the unit should update its weights accordingly. 

To update the weight vector <p (17), the following procedure is used. To each input 

vector is associated an expected reinforcement value. Since these two parameters are 

available, to learn their association, a supervised learning paradigm can be used. In 

this case the Widrow-Hoff LMS rule is chosen. The second important point in the 
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learning rule (16) is that the standard deviation depends on the expected 
reinforcement. Therefore, the SRV unit can control the extent of search through the 
standard deviation value. In fact, as the expected reinforcement increases, the standard 
deviation decreases and, therefore, the size of the search space is narrowed in the 
neighborhood of the mean output. 

4.3 Integration of the layer of SRV units in the HCNN 

Recalling that the HCNN is composed of two layers of neurons using backpropagation 
(BP) and one output layer with SRV neurons, the input vector of the SRV units is the 
output vector of a hidden layer of BP units. To train the BP neurons, an error vector is 
needed and since SRV output units are used, the error signal is not available because 
there is no desired output. (Gullapalli, 1990) wrote that randomly perturbing the mean 
output and observing the consequent change in the evaluation, enables the unit to 
estimate the gradient of the evaluation with respect to the output. Therefore, to train the 
backpropagation layers, the actual error is replaced with an estimated error of the 
following form: 

^srv = (r k -r k ){a k -M k ) (18) 

<*k 

In order to propagate the error from the SRV units back to the BP units, we have used the 
following equation. 

dBP =@ T,jSRV (19) 

Where 0[i, j] is the weight #used to compute the mean parameter of the z th SRV unit from the 7 th 
BP unit's output (considered as input; of the SRV unit). With this properly propagated error (19), 
we can train the BP layers using the standard backpropagation algorithm. 

The purpose of the following sections is to study the performances of the proposed model. 
In a first part, the model is applied to define the posture of the MANUS manipulator used in 
rehabilitation robotics. Then, we evaluate the capability of the model to construct the 
posture of an anthropomorphic upper-limb model for a particular grasp with the 
incorporation of noise in the reinforcement signal and obstacles in the environment. 

5. Simulation results 

The purpose of this section is to study the performances of the proposed model. In a first 
part, we evaluate the capability of the model to construct hand posture for a particular grasp 
with the incorporation of noise in the reinforcement signal. In a second time, for three 
different tasks, proposed model is tested. 

5.1 Grasping with noise and uncertainty 

In this section, simulation results are presented. The hand posture to grasp a 
parallelepiped with five fingers is constructed. Before each test, the weights of the 
HCNN are initialized with random values over the interval [-0.5, 0.5] and the hand 
palm is placed in a random configuration within the search space. The learning is 
performed until a reinforcement greater than 0.99 is obtained or until the maximum 
number of iterations is reached. 
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2000 0,001 




a/ b/ 

Fig. 3. Evolution of learning parameters according to noise level: a/ Reinforcement signal 
and b/ corresponding error (m). 

In order to identify the influence of uncertainty and noise reflecting low quality sensors, we have 
considered four levels of noise with zero mean and standard deviation <j n of 0.2, 0.1, 0.05 and 
(which corresponds to a deterministic reinforcement, n e [0, 1]). Each trial has duration of 2000 
steps and the bounds of the workspace are defined in Table I. In order to have a satisfying 
convergence, we use the following learning rates oqv = 0.01 for the BP units, a = 0.01 and p = 0.01 
for the SRV units. In Fig. 3a/, the obtained reinforcement signals with the four noise standard 
deviation levels are displayed as well as the corresponding deterministic error in Fig. 3b/. The 
algorithm finds a satisfactory solution even if the curves are more irregular for large standard 
deviations. We can also notice that the convergence is of the same order than the trial with 
deterministic reinforcement attesting the model robustness to noise. Finally, a satisfactory solution 
is obtained after a relatively low number of time steps. 

At the beginning of the learning process the SRV units expected reinforcement is low and the 
standard deviation parameter is high; thus the exploratory behavior is predominant. As training 
proceeds, the mean of the SRV units Gaussian distribution is gradually shifted in such a direction 
that the reinforcement increases. Consequently, the expected reinforcement increases and the 
standard deviation decreases rapidly as well as the total error. Then, the exploitation behavior 
becomes predominant and the hand configuration is slightly modified to optimize the solution. 





X(m) 


Y(m) 


Z(m) 


0KC) 


0Y(°) 


ezn 


min. value 


-0.1 


-0.1 


0.15 


-45 


-45 


-135 


max. value 


0.1 


0.1 


0.25 


45 


45 


-45 



Table 1. Search space bounds for the rectangular block five fingered grasp. 

The results of the methods are constructed for grasps with different number of fingers and different 
contact sets. MATLAB environment is used to solve the grasp planning problem (Pentium 4 HT 
3.0Ghz, 512 ram, FSB 800 MHz). Three different grasps are considered in increasing number of 
fingers : grasping a glass with three fingers (task 1) or four fingers (task 2) and grasping a videotape 
with five fingers (task 3) (Fig. 4). For each task, twenty simulations are performed using the 
proposed model. Only the deterministic reinforcement with a zero noise level is considered. The 
results are summarized in Table 2. Results are displayed for each task in three columns. For task 1, 
in the first and second ones are indicated the total error and the detail for each fingertip after 2 and 5 
seconds of learning respectively. Finally, in the third column, the error after 2000 iterations is 
displayed. For task 2, results are provided after 2 and 6 seconds of learning in first and second 
columns and after 1000 iterations in the third one, for task 3, after 3 and 7 seconds and after 1000 
iterations in column 1, 2 and 3 respectively. Average values ± standard deviation are provided. 
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Taskl 


Task 2 


Task 3 


Time (s) 


2 


5 


18.5+12.3 


2 


6 


14.8+5.2 


3 


7 


18.9+3.2 


Error (mm) 


9.0+9.4 


4.1+1.1 


3.1+0.04 


8.4+5.8 


6.2+1.7 


5.1+1.2 


21.3+21.7 


9.1+2.8 


6.9+1.3 


Thumb (mm) 


2.6+1.4 


1.9+0.8 


1.5+0.7 


3.4+2.9 


2.5+1.3 


2.1+0.9 


7.0+9.8 


3.7+1.5 


3.0+0.9 


Index (mm) 


4.5±7.5 


1.1+0.7 


0.8+0.4 


1.9+1.5 


1.3+0.3 


1.2+0.4 


4.9+6.2 


2.2+1.3 


1.6+0.9 


Middle (mm) 


2.0+2.2 


1.1+0.4 


0.8+0.4 


1.5+1.5 


1.2+0.7 


0.9+0.5 


3.5+5.1 


1.1+0.5 


1.0+0.3 


Ring (mm) 








1.6+0.9 


1.2+0.5 


0.9+0.5 


2.2+2.8 


1.0+0.5 


0.7+0.3 


Pinky (mm) 














3.7+7.4 


1.2+1.0 


0.7+0.3 



Table 2. HCNN results. 

Finally, in Fig. 4, the obtained postures for task 1, 2 and 3 are displayed. 




a/ Task 1 b/ Task 2 c/ Task 3 

Fig. 4. Postures obtained by HCNN and optimization for the three tasks: a/ glass grasp with 
3 fingers, b/ glass grasp with four fingers, c/ videotape grasp with five fingers. 
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a/ b/ 

Fig. 5. Complete arm model, Model architecture when considering the arm and obstacles in 
the environment. 



6. Complete upper-limb posture definition with obstacle avoidance 

In this section, a complete upper-limb is considered including an arm model associated with 
the hand model described and used in the former section. The model capabilities are 
extended by considering the presence of unknown obstacles in the environment (Rezzoug & 
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Gorce, 2006). The resolution principle is the same than the one used when considering the 
hand only. Instead of the hand palm position, the arm joints configuration is considered as 
input of the ACNN (Arm Configuration Neural Network) which replaces the HCNN of the 
original model. Also, the arm forward model is used to compute the contact set in the frame 
of the considered fingers as shown in Fig. 5b/ . 

6.1. Arm model 

The model of the arm is composed of two segments and three joints (Fig. 5a/). The first 
joint, located at the shoulder (gleno-humeral joint) has three degrees of freedom. The second 
joint is located at the elbow and has one degree of freedom. Finally, the last joint, located at 
the wrist, has three degrees of freedom. The final frame of the last segment defines the 
orientation of the hand palm. According to this formulation, the arm posture is completely 
defined by the joint angles vector q = [qi , c\i, c\$, cj4, c\5, Cj6/ ^z] T . The chosen model has 7 
degrees of freedom (Fig. 5a/). 




200 400 600 800 1000 1200 1400 1600 1800 2000 
Iteration number 



Fig. 6. Evolution of the reinforcements Ri, R2, R3 and R4 relative to the iteration and collision 
numbers. 



6.2. Improving learning performances by shaping 

In order to define a suitable reinforcement signal, two aspects of the performance have to be 
taken into account. The first one evaluates the upper-limb positioning task while the second 
is relative to collision avoidance. In the following, the different steps that conduct to the 
definition of the appropriate reinforcement signal are described. Firstly, the positioning task 
is evaluated. To do this, given the arm and fingers configuration, the actual position of the 
fingertips is calculated using forward kinematics. 

The simplest form of the reinforcement Ri as used in section 4 gives a maximum penalty if 
error Ek is large and is given in (14) and recalled here (a is a positive real number): 



R l =\-h(a.E k ) 



(20) 



Starting from the definition of Ri, the basic expression of the reinforcement signal that 
incorporates collision avoidance behavior is given by: 
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^i if no collision (21) 

R, 1 2 if collision 

In order to fulfill the secondary task i.e. collision avoidance, the reinforcement Ri is 
divided by two whenever a collision is detected. Therefore, even if the principal task is 
accomplished with success the reinforcement is low due to the occurrence of a collision. 
One can notice the simplicity of the incorporation of collision avoidance behavior in the 
learning process. However, the criterion R2 uses a somewhat crude strategy and the 
results may not be as satisfying as expected. Indeed, the learning agent has to directly 
discover the right strategy to satisfy two kinds of constraints at the same time. This is a 
more complex task than arm positioning only. 

In order to circumvent this difficulty, we propose to use a technique inspired from animal 
training called shaping (Skinner, 1938). (Gullapalli, 1992) gave a nice definition of this 
concept and applied it to the frame of reinforcement learning : "The principle underlying 
shaping is that learning to solve complex problems can be facilitated by first learning to 
solve simpler problems. . . . the behavior of a controller can be shaped over time by gradually 
increasing the complexity of the task as the controller learns ". 

To incorporate shaping in the learning procedure, the basic idea is to let the agent 
learn the positioning task first and the collision avoidance behavior during a second 
phase. To implement this, a reinforcement signal that gradually increases over time the 
penalty due to collisions is defined. In this way, the agent can learn adequately the 
first task and modify its behavior in order to achieve the second one. The 
reinforcement value used in this case is the following (i is the current iteration number 
and p the maximum number of iterations): 

{R { if no collision (22) 

Rj(\ + ilp) if collision 

If collisions occur, for the same value of Ri, an increase of i conducts to an increase of the 
denominator in (22) and consequently to a decrease of R3. If i = p, we can notice that R3 = R2 
and that there is a gradual shift from Ri (no penalty for collision) to R2 (full penalty for 
collision). This weaker definition of arm positioning with collision avoidance may be easier 
to learn than direct collision avoidance as defined by R2. The evolution of R3 with Ri = 0.99 
when collisions occur is displayed in Fig. 6. 

The main drawback of R3 is that the same penalty is applied whatever the number of 
collisions. It may be easier to learn the task successfully if the learning agent can grade 
differently two situations with different numbers of collision, giving more penalty to the 
posture conducting to more collisions or interpenetrations. In order to solve this problem, 
we define the reinforcement R4: 

^1 if no collision (23) 

4 ~ [^ l(\ + c P (Up)) if collision 

where c is the number of detected collision(s) or interpenetration(s) and /?a positive real number. 
Reinforcements R3 and R4 use the same strategy, except that R4 takes into account the number 
of collisions. Indeed, for the same value of Ri, i and p, an increase of c conducts to an increase 
of the denominator in (23) and therefore to a decrease of the reinforcement R4. If c = 1, we 
notice that R4 = R3. The evolution of R4, with different values of c is displayed in Fig. 6. 
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Fig. 7. Environments for the two grasping tasks. 

6.3. Simulation results 

The task to be performed is to grasp a cylinder with three fingers. Two different 
environments are considered, the first one with a big obstacle between the arm and the 
object and the second one with two obstacles (Fig. 7). 



Reinforcement 


Ri 


R 2 


R 3 


R4 


Success 


26 


20 


22 


26 


Causes 
of failure 


Positioning task 


4 


10 


7 


4 


Collision 


22 


3 


6 


2 


Mean iterations number 


102 


430 


281 


310 


Standard deviation 


126 


271 


259 


245 



Table 3. Simulation results for Task 1. 



Reinforcement 


Ri 


R 2 


R 3 


R4 


Success 


25 


8 


22 


16 


Causes 
of failure 


Positioning task 


5 


22 


7 


14 


Collision 


24 


4 


2 


5 


Mean iterations number 


120 


454 


273 


260 


Standard deviation 


161 


266 


228 


223 



Table 4. Simulation results for Task 2. 

Thirty simulations are performed for each reinforcement and for each environment. The 
weights of the ACNN are initialized with random values over the interval [-0.5, 0.5] and a 
random arm configuration is chosen within the search space. We use the following values 
for the parameters of the ACNN : a B p = 0.03, cc S rv = 0.03, p = 0.01, m = 14, n 2 = n 3 = 20, n 4 = 7, 
a = 1.5, p = 2000 and j5 = 0.25 The learning has to be completed for each task and 
environment and is performed until a reinforcement greater than 0.99 is obtained or until 
the maximum number of iterations is reached. A FCNN is constructed off line for each 
finger before the simulations. Collision or interpenetration check is implemented with a two 
steps scheme. Axis aligned bounding boxes are constructed for each element of the 
environment to make a first check. If it is positive, the distance between any pair of solids 
that are likely to collide is computed. This is done by minimizing the distance between any 
pair of points on the surface of two elements of the scene modeled with superquadrics. 
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In Tables 3 and 4 are displayed the obtained results. In the first row, the number of successes 
is indicated for each reinforcement. This corresponds to the case where the reinforcement is 
greater than 0.99. In the second and third rows is indicated the number of cases for which a 
failure is due either to the positioning task or to collisions. Finally, for the successful cases, 
the last two rows indicate the mean and standard deviation of the required number of 
iterations to obtain a suitable reinforcement. 
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Fig. 8. Examples of learning parameters evolution for task 1 obtained with reinforcement a/ 
Ri , b/ R 2 , c/ R 3 and d/ R4. 

Reinforcement Ri is used as a reference in order to demonstrate that the other 
reinforcements R2, R3 and R4 have effectively an effect on collision avoidance. 
The first observation is that the incorporation of collision avoidance behavior in the 
reinforcement signal effectively leads to collision avoidance even if the positioning task is 
not achieved. Using Ri, 22 solutions out of 26 valid ones are obtained with collisions 
between the upper limb and the environment for the first task and 24 out of 25 for the 
second task. This number falls to 3, 6 and 2 for R2, R3 and R4 for the first task and 4, 2 and 5 
for the second task respectively. Also, one can notice that there is an increase of the number 
of successes when shaping is incorporated compared to the case where crude collision 
avoidance reinforcement is used (R2). This is particularly obvious for the second task (8 
successes with R2 compared to 22 using R3). This suggests that the strategy of shaping 
conducts to a higher number of valid solutions and therefore that the learning is enhanced. 
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In Fig. 8 is presented the evolution of the learning parameters using reinforcements Ri, R2, R3 and 
R4. These curves give interesting information on the learning process. Indeed, using Ri (Fig. 8a/), 
the evolution of the expected reinforcement which is an internal parameter representing the 
learning performance is "monotonic" and increasing, suggesting that the learning is performed in 
one sequence and that there is a gradual shift from exploration to exploitation. This parameter 
using R2 (Fig. 8b/) has a far less regular evolution, denoting the increased difficulty to find a 
solution due to the strict nature of R2. The evolution of R3 and R4 (Fig. 8c/ and 8d/) show the effect 
of shaping. At the beginning of the learning, the expected reinforcement follows the same pattern 
than using Ri, i.e. a monotonic increasing function on average. As the number of iterations (NOI) 
increases, the occurrence of collision(s) is more penalized and there is a decrease of the expected 
reinforcement. This causes an augmentation of exploration evidenced by the augmentation of the 
standard deviation (bold dashed curve). Then, taking advantage of the previous learning of the 
positioning task, a solution is found that exhibits collision avoidance behavior. This is particularly 
visible in Fig.9 where the evolution of the number of collisions using R3 is shown (this case 
corresponds to the results of Fig. 8c/). The agent has learned the first task without considering the 
collisions. This is evidenced by the high number of collisions at the beginning of the learning. 
Then, as the number of iterations increases this behavior is more penalized and using the previous 
acquired knowledge the agent is able to learn the second aspect of the task (collision avoidance) 
without a deterioration of the performance concerning the first aspect (i.e. grasping). 




Fig. 9. Evolution of the number of collision(s) using reinforcement R3. 

To determine if the use of the different reinforcements has an effect on the NOI, a one way analysis 
of variance (ANOVA) (Miller, 1997) on the number of iterations to complete the task is conducted. 
A Bonferoni post-hoc test is used to perform multiple comparisons between means. The ANOVA 
evidences a significant difference between four groups means (F(3, 161) = 14.33, p<0.0001). Also, the 
post-hoc tests show a significant increase of the NOI using R2 compared to the NOI using R3 and R4 
(p<0.05). Also, a significant increase of the NOI using R2, R3 and R4 compared to the NOI using Ri is 
noticed (p<0.05). There is no significant difference between the NOI using R3 and R4. These results 
suggest that learning the positioning task is easier than the positioning task with collision avoidance 
because, on average, more iterations are needed whatever the chosen reinforcement. Secondly, the 
incorporation of shaping in the learning process reduces significantly the required number of 
iterations to reach the goal. Finally, taking into account the number of collisions in the reinforcement 
definition does not seem to improve significantly the learning performances. Therefore, among all 
the reinforcement signals proposed in this study, we can consider that R3 is the best one to perform 
grasping posture definition with obstacles in the frame of the considered model. In Fig. 10, the 
posture obtained to grasp a cylinder surrounded by 3 obstacles is shown. 
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Fig. 10. Upper-limb posture to grasp a cylinder surrounded by 3 obstacles. 

In the following section, a non anthropomorphic arm is considered. The method is slightly 
modified to handle this case. 

7. MANUS robot configuration definition 

Several movements are needed to grasp an object: an approach phase during which the 
end-effector is brought in the vicinity of the object followed by a grasping phase that 
implies precise adjustments in order to orient properly the end-effector. This latter 
phase can necessitate fine motions that are tedious if executed in manual control. To 
reduce the difficulty of this task, we propose to automate partially the grasping phase 
working in the vicinity of the arm configuration chosen by the user at the end of the 
approach phase. More precisely, we define the angular configuration of the robot joints 
in order to place the end-effector in an adequate configuration. 
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Fig. 11. a/ Degrees of freedom of the MANUS robot , b/ Model architecture. 



7.1 . Robot arm model 

To define the mobile platform configuration 9 degrees of freedom have to be controlled (6 for the 
MANUS and 3 for the mobile base). This task is made difficult due to the existence of an infinite 
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number of solutions to put the platform and the MANUS arm in a given configuration (Fig. 11a/). 
In order to identify the object to be grasped, it is necessary to obtain information from a 
camera. It seems important to define the needed amount of information to achieve the task 
considering the trade off between efficacy and complexity. 

During the grasping phase, two points defined on the end-effector of the MANUS arm 
are associated with two points on the surface of the object. The goal of the grasping 
task is to bring the MANUS arm in such a configuration that the two pairs of points 
overlap. In this way, the constraints relative to the position as well as the orientation of 
the end-effector are treated. Furthermore, the amount of the needed information is 
limited to the position of four points in 3D. The corresponding model architecture is 
displayed in Fig. lib/. 

The inputs of the model are the location of the two points of interest both on the MANUS 
gripper and on the object and also the arm joint limits. The output is the mobile platform 
configuration that optimizes a particular cost function composed of two parts: 

1. A first term that insures that the axes defined by the two points on the surface of the 
object and on the MANUS gripper are collinear. 

2. A second term to minimize the distance between each couple of points. 

The first part evaluates the orientation of the gripper relative to the object. Its expression is 
given in (24). 

R =ln n I (24) 

iV l ^Gripper "Object | v ' 

nGripper is the unit vector aligned with the axe defined by the two points on the MANUS 
gripper and nobject is the unit vector defined by the two points on the object. The maximum 
value of Ri is reached when the two vectors are collinear, then Ri = 1. On the other hand, 
when the two vectors are orthogonal Ri = 0. 

The second part of the cost function evaluates the distance between the couples of points. It 
is defined as following: 

R 2 =l-tanh(d) ( 25 ) 

where 

d = min(d l ,d 2 ) ( 26 ) 

With 

7 llvObject ^-Gripper II . I |y Object ^-Gripper II C2.7) 

j HvObject ^-Gripper || . || ^Object ^-Gripper II (28^ 

x object anc [ X Gri PP er ([ = \ t 2) represent the 3D position of the points attached to the object and to 

the gripper in. d represents the minimum of the summed distances between the couple of 
points on the gripper and the object. 

The function tank insures that Ri lies in the interval [0, 1]. R2 is minimum if d is high and 
maximum (R2 = 1) if d = 0. Combining the two criteria, the cost function R (29) is obtained. 

* = ^±*l (29) 
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Fig. 12. a/ Criterion evolution and b/ simulated MANUS robot configuration. 

7.2. Simulation results 

In this section, simulation results are presented. The grasp of a cube placed at different 
locations is considered. The contact points location and the position of the object are known 
and considered as input data. The arm starts from a random position within its workspace. 
The graphs of Fig. 12a/ displaying the evolution of the learning criteria Ri, R2 and R during 
learning show the ability of the model to discover the adequate configuration while 
satisfying orientation and position constraints. Also, to evaluate the performances of the 
method over a workspace, 30 simulations are performed for each of 64 object positions 
equally distributed in a 0.6 m x 0.6 m x lm workspace. The obtained mean d (3) is 9.6 + 4.1 
mm. For one situation, the platform configuration is shown in Fig. 12b/. 



8. Conclusion 

In this chapter, a new model was proposed to define the kinematics of various robotic 
structures including an anthropomorphic arm and hand as well as industrial or service 
robots (MANUS). The proposed method is based on two neural networks. The first one 
is dedicated to finger inverse kinematics. The second stage of the model uses 
reinforcement learning to define the appropriate arm configuration. This model is able 
to define the whole upper limb configuration to grasp an object while avoiding 
obstacles located in the environment and with noise and uncertainty. Several 
simulation results demonstrate the capability of the model. The fact that no 
information about the number, position, shape and size of the obstacles is provided to 
the learning agent is an interesting property of this method. Another valuable feature 
is that a solution can be obtained after a relatively low number of iterations. One can 
consider this method as a part of a larger model to define robotic arm postures that 
tackles the "kinematical part" of the problem and can be associated with any grasp 
synthesis algorithm. In future work, we plan to develop algorithms based on 
unsupervised learning and Hopfield networks to construct the upper-limb movement. 
In this way, we will be able to generate an upper-limb collision free trajectory in joint 
coordinate space from any initial position to the collision free final configuration 
obtained by the method described in this article. 
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1. Introduction 



This chapter discusses the advantages and feasibility of using compliant actuators in 
exoskeletons. We designed compliant actuation for use in a gait rehabilitation robot. In such 
a gait rehabilitation robot large forces are required to support the patient. In case of post- 
stroke patients only the affected leg has to be supported while the movement of the 
unaffected leg should not be hindered. Not hindering the motions of one of the legs means 
that mechanical impedance of the robot should be minimal. The combination of large 
support forces and minimal impedances can be realised by impedance or admittance control. 
We chose for impedance control. The consequence of this choice is that the mass of the 
exoskeleton including its actuation should be minimized and sufficient high force 
bandwidth of the actuation is required. Compliant actuation has advantages compared to 
non compliant actuation in case both high forces and a high force tracking bandwidth are 
required. Series elastic actuation and pneumatics are well known examples of compliant 
actuators. Both types of compliant actuators are described with a general model of 
compliant actuation. They are compared in terms of this general model and also 
experimentally. Series elastic actuation appears to perform slightly better than pneumatic 
actuation and is much simpler to control. In an alternative design the motors were removed 
from the exoskeleton to further minimize the mass of the exoskeleton. These motors drove 
an elastic joint using flexible Bowden cables. The force bandwidth and the minimal 
impedance of this distributed series elastic joint actuation were within the requirements for 
a gait rehabilitation robot. 

1.1 Exoskeleton robots 

Exoskeletons are a specific type of robots meant for interaction with human limbs. As the 
name indicates, these robots are basically an actuated skeleton-like external supportive 
structure. Such robots are usually meant for: 

• Extending or replacing human performance, for example in military equipment 
(Lemley 2002), or rehabilitation of impaired function (Pratt et al. 2004), 

• Interfacing; creating physical contact with an illusionary physical environment or 
object; these haptic devices are usually referred to as kinaesthetic interfaces. Possible 
applications appear for example in gaming and advanced fitness equipment, or in 
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creating 'telepresence 7 for dealing with hazardous material or difficult circumstances 
from a safe distance (Schiele and Visentin 2003). 

• Training human motor skills, for example in the rehabilitation of arm functionality 
(Tsagarakis and Caldwell 2003) or gait (Colombo et al. 2002) after a stroke. 

Every typical application has specific demands from a mechatronical design viewpoint. Robots in 
interaction with human beings should be perceived as compliant robots, i.e. humans should be 
able to affect the robots motions. In other words, the soft robots should have an impedance (or 
admittance) control mode, sensing the actions of the human beings. Since the impedance or 
admittance control does not need to be stiff but compliant we can use compliant actuators. An 
advantage of compliant actuators is that the impedance at higher frequencies is determined by the 
intrinsic compliance of these actuators. For non compliant actuators the impedance for frequencies 
higher than the control bandwidth is determined by the reflected motor mass. In general, the 
impedance of the reflected motor mass will be much higher than intrinsic compliance of compliant 
actuators, especially when the needed motor torques and powers will be high. The advantage of 
compliant actuators is that they not only have a favourable disturbance rejection mode (through 
the compliance), but also have sufficient force tracking bandwidth. The compliant actuators 
discussed in this article were evaluated for use in an exoskeleton for gait training purpose, but 
might find wider application. First of all the specific application will be described, followed by 
models and achieved performance of the actuators. 

1.2 Context: a gait rehabilitation robot 

We are developing a LOwer-extremity Powered ExoSkeleton (LOPES) to function as a gait 
training robot. The target group consists of patients with impaired motor function due to a 
stroke (CVA). The robot is built for use in training on a treadmill. As a 'robotic therapist 7 
LOPES is meant to make rehabilitation more effective for patients and less demanding for 
physical therapists. This claim is based on the assumptions that: 

• Intensive training improves both neuromuscular function and all day living 
functionality(Kwakkel et al. 2002; Kwakkel et al. 2004), 

• A robot does not have to be less effective in training a patient than a therapist 
(Reinkensmeyer et al. 2004; Richards et al. 2004), 

• A well reproducible and quantifiable training program, as is feasible in robot assisted 
training, would help to obtain clinical evidence and might improve training quality 
(Reinkensmeyer et al. 2004). 

The main functionality of LOPES will be replacing the physiotherapists' mechanical 
interaction with patients, while leaving clinical decisions to the therapists' judgment. The 
mechanical interaction mainly consists of assistance in leg movements in the forward and 
sideward direction and in keeping balance. 

Within the LOPES project, it has been decided to connect the limbs of the patient to an 
'exoskeleton' so that robot and patient move in parallel, while walking on a treadmill. This 
exoskeleton (Fig. 1) is actuated in order to realize well-chosen and adaptable supportive actions 
that prevent fail mechanisms in walking, e.g. assuring enough foot clearance, stabilizing the knee, 
shifting the weight in time, et cetera. A general aim is to allow the patient to walk as unhindered as 
possible, while offering a minimum of necessary support and a safe training environment. 
Depending on the training goals, some form of kinaesthetic environment has to be added. This 
constitutes the main difference between LOPES and the commercially available gait-trainers. 
Those are either position controlled devices that overrule the patient and/ or allow only limited 
motions due to a limited number of degrees of freedom, and/ or are not fully actuated (Hesse et al. 
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2003). Position controlled devices omit the training challenges of keeping balance and taking 
initiative in training. More research groups have recognized this shortcoming (Riener et al. 2005). 
In the control design of the exoskeleton in general two 'extreme' ideal modes can be defined, that 
span the full range of therapeutic interventions demanded in the LOPES project. In one ideal mode, 
referred to as 'robot in charge', the robot should be able to enforce a desired walking pattern, defined 
by parameters like walking speed and step-length. This can be technically characterized as a high 
impedance control mode. In the other ideal mode, referred to as 'patient in charge' the robot should 
be able to follow the walking patient while hardly hindering him or her. This can be technically 
characterized as a low impedance control mode. An intelligent controller or intervention by a 
therapist then can adjust the actual robot behaviour between these high and low impedance modes. 



Ili:*:ikki:::ur. 






Fig. 1. Design of LOPES. Left: DoFs of the exoskeleton that are actuated. Other DoFs are left free 
(ankle knee ab/aduction) or constrained (hip and knee endo/exorotation). middle: schematic 
drawing of the construction for connecting the exo-skeleton to the fixed world consisting of height 
adjustable frame (1), two sets of parallel bars with carriages for the for-/backward (2) and 
sideways (3) motion that are both actuated, and a parallogram with weight compensation the 
allows for vertical pelvic motion as occurs while walking. Right: photo of LOPES. 



1.3 Impedance- versus admittance-controlled interactive robots 

Many issues arising in the design of interactive neuro-rehabilitation robots are similar to 
those appearing the field of haptic, or more precise: kinaesthetic robotics (Brown et al. 2003). 
The characteristic feature of these robots is the bi-directionality of being able to both 'read 
from' and 'write to' a human user (Hay ward and Astley 1996). Such robots are in general 
meant to display virtual objects or a virtual environment to a user. This user then can 
interact with a virtual or distant 'world' in a mechanically realistic way. 
In contrast, interactive neuro-rehabilitation robots are meant to operate in between both 
stated modes of 'robot in charge' and 'patient in charge', acting as a 'robotic therapist'; not 
to display virtual objects as realistic as possible. Another difference, specific for limb- 
guiding exoskeletons, is that a kinaesthetic display usually has an end-effector that displays 
the information at one location on the robot, while an exoskeleton necessarily interacts at 
several points with human limbs as it is connected in parallel to the limbs. This implies that 
not only an end-effector force is important, but all 'internal' torques over all actuated joints. 
This all makes it necessary to first select the optimal basic control outline for this kind of 
robot, as different outlines imply different robot construction and actuator demands. 
In general there are two basic ways to realize a kinaesthetic display (Adams and Hannaford 
2002); impedance-control-based that is 'measure position and display force' (Fig. 2 left) and 
admittance-control-based (Fig. 2 right) that is 'measure force and display position', although 
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hybrid forms exist. The important difference is that in impedance-control the quality of the 
display will depend on the accuracy of the position sensors and the bandwidth and accuracy of 
the force servos, and in case of admittance-control on the accuracy of the force sensors and the 
bandwidth and accuracy of the position servo. The bandwidth of the mentioned servos will 
depend on both the robot construction and the actuators. The choice between high performance 
force servo quality and high performance position servo quality puts specific demands on the 
robot construction and actuation. This choice has to be made even if a hybrid control architecture 
with both position and force sensing is used. To make clear what this choice basically is about it 
can also be presented as the choice between a lightweight (and flexible) uncompensated 
construction and a rigid (and heavy) controller-compensated construction. 
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Fig. 2. Basic outline of an impedance-control (left) and admittance-control based rehabilitation robot. 

A fundamental limitation of impedance control is that in each specific controlled degree of 
freedom (or 'dof) the dynamical behavior of the robot construction in this 'dof appears in the 
force transfer ('device dynamics') (Fig. 1). It can only be compensated for in case of a proper 
dynamical predictive model, and proper measurements of position and velocity for stiffness and 
friction compensation respectively. Mass compensation is possible to a small extent only, by 
adding a force or acceleration feedback loop (resulting in a hybrid control architecture). Best 
results within an impedance-control architecture are obtained using a lightweight, low friction 
construction and a low impedance actuator, so that the intrinsic mechanical impedance of the 
device is kept low. Impedance controlled robots typically display 'low-forces 7 and lack 
performance in kinaesthetic display of high inertia and high stiffness. (Linde and Lammertse 
2003). This is not essential in gait-rehabilitation as no stiff contacts or large inertias have to be 
displayed, although relatively high forces certainly appear. 

An admittance control scheme, on the other hand, demands for a high positioning bandwidth, 
and therefore for high(-er) power actuators and a stiff construction without backlash (Linde and 
Lammertse 2003). A limitation of admittance control is that the display of low stiffness ('free 
motion') can become instable, especially in case of a rather stiff user, because of the high control 
gains needed in this case. Another limitation is that admittance can only be controlled at the 
location(-s) of the force sensors. The construction will display its intrinsic impedance at other 
locations, since the interaction behaviour there cannot be influenced by the controller without 
force sensing. This might pose a problem for application in neuro-rehabilitation, because of the 
possible safety threat (for instance for the therapist). The robot will not react to mechanical 
interactions that by-pass the force sensors. This is all the more important as with admittance 
control actuators and construction will be considerably heavier. 

Summarizing, compared to general kinaesthetic devices a less critical stiffness- and mass- 
display is demanded and movements are relatively slow. An admittance controlled system 
imply larger safety threats due to the higher required actuator power, rigidity and inertia of 
the robot and is less stable when a low stiffness is displayed. Considering all this we choose 
an impedance-control based design strategy for our exoskeleton. An important advantage is 
also that the programmed dynamical behaviour will be available everywhere on the 
construction, that is, in every actuated degree of freedom. 
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Impedance control implies that the actuators should be perfect force sources, or, realistically, low- 
impedance, high precision force sources (in contrast with position sources that would be needed 
for admittance control). The construction therefore should be lightweight, although being able to 
bear the demanded forces, and should contain less friction. Fig. 3 shows the robot control outline 
for one degree of freedom (DoF) that is impedance controlled. 
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Fig. 3. Schematic control outline for one degree of freedom. The inner loop (shaded area) assures 
that the actuator acts as a pure force source. The outer loop (impedance controller) sets the force 
reference based on desired impedance and actual displacement. The load reflects the 
robot/ human-combination dynamics in the considered DoF. In case the actuator is compliant the 
load displacement directly influences the force output. Symbols: xi oa d - position load; F - force; 
Fdis - external force disturbance;^ - force sensor noise; n x - position sensor noise. 



1.4 Advantages of compliant actuation in impedance controlled devices with 'high 
force' demands 

As addressed by Robinson (Robinson 2000), commonly used actuators are poor force 
actuators, even if in many theoretic approaches actuators are supposed to be pure force 
sources. For small robots common brushless DC motors in combination with cable drives, 
usually suffice, as for example used in the WAM-arm (Townsend and Guertin 1999), a 
haptic device. In general, actuators with high force and high power density typically have a 
high mechanical output impedance due to necessary power transmission (e.g. geared EM 
motors) or the nature of the actuator (e.g. hydraulics). These actuators do not have a large 
force tracking bandwidth in case high forces have to be displayed. 

The solution suggested by Robinson, and several others (Morrell and Salisbury 1998; 
Robinson 2000; Bicchi et al. 2001; Sugar 2002; Zinn et al. 2004) is to decouple the dynamics of 
the actuator and the load, by intentionally placing a compliant element, e.g. a spring, 
between both. The actuator can then be used in a high gain position control loop, resulting 
in an accurate control of spring deflection and resulting force, while compensating the 
internal dynamics of the actuator. The advantage of a compliant element is that it not only 
filters friction and backlash and other non-idealities in transmission drives from the force 
output, but also that it absorbs shock loadings from the environment. 

An alternative way of realizing such a compliant actuator is to use a pneumatic actuator, 
which is compliant due to the physics of air. The most straightforward way is by using a 
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double-acting cylinder, but also an antagonistic pair of fluidic muscles achieves a double 
acting system (Ferris et al. 2005). Instead of measuring spring-deflection, pressure 
measurements can be used as indirect force measurement. 

Mechanical compliance in the actuation does not offer its advantages without costs. The lower the 
stiffness, the lower the frequency with which larger output forces can be modulated. This is 
caused by saturation effects that limit the maximal achievable acceleration of the motor mass and 
spring length, or by the limited fluid flows for given certain pressures. A limited Targe force 
bandwidth 7 decreases in turn the performance in terms of positioning-speed and -accuracy. A 
careful design and trade-off is needed to suit such actuators for a typical application. 
As mentioned by Robinson, it is not yet obvious how to select the proper elastic actuator for 
a certain application. His general models appeared to be useful for predicting behaviour of a 
series elastic actuator configuration, but do not clarify what the limitations of several 
implementations of elastic actuators are. He suggested that important parameters would be 
the force and power density levels of the specific configuration. To make a proper 
comparison between series elastic actuators and pneumatic elastic actuators, the latter will 
have to be described in similar parameters as the first. Once this is achieved the question of 
the limits and (dis-) advantages of both types of elastic actuation can be addressed. 

1.5 Actuator demands in an impedance controlled rehabilitation robot 

Impedance control implies specific demands for the actuators in the robot. They should: 

be 'pure 7 (low impedance) force sources 

add little weight and friction to the moving robot construction in any degree of 

freedom, not only the specific degree of freedom actuated by the considered actuator 

be safe, even in case of failure 

allow fast adjustment to the individual patient's sizes 

be powerful enough for the 'robot in charge 7 task. 
More specific, we decided that the actuators should be able to modulate their output force with 
12 Hz for small forces, and 4 Hz for the full force range. Maximum joint moments differ per joint, 
and range from 25 to 60 Nm. Joint powers range up to 250 Watt per joint. These numbers were 
based on study of the human gait cycle and the motion control range of a human therapist. An 
analysis of a nominal gait cycle (Winter 1991) was studied to obtain maximal needed torques, 
speeds and powers during walking, and general data on human motor control were studied to 
estimate maximum expected force-control speed and accuracy of a physical therapist. 
The resulting actuator bandwidths are typically lower, and the forces typically higher than in the 
specifications of common kinaesthetic devices which are intended for haptic display of a virtual 
object, but not to assist humans. Actuators usually selected for kinaesthetic devices are either 
heavy (like direct drive electro-motors) or poor force actuators (like geared DC motors). 
We evaluated different compliant actuators. A series elastic actuator and a pneumatic cylinder 
were dimensioned to meet the required demands state above. Both type of actuators were 
modelled, evaluated and compared with each other. Next, we also designed and evaluated a 
new type of series actuator. To minimize the mass of the exoskeleton we disconnected the motor 
from the exoskeleton and connected the motor with the elastic joints through Bowden cables. 

2. A framework for compliant actuation 

In this paragraph we first will derive a general model for compliant actuators. Next we will 
put pneumatic and series elastic actuation into this general framework. 
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2.1 General model of a compliant actuator 

The first step in comparing compliant actuators is a general model (Fig. 4) that is detailed and 
flexible enough to describe the essential behaviour of any kind of compliant actuator, in this case 
both a series elastic actuator based on a brushless DC motor, and a double-acting pneumatic 
cylinder. The basic element of such an actuator is its elastic element or elasticity that we define as 
it intrinsic stiffness (K s ). Furthermore it should contain a motor block that modulates spring 
length or the force. A last important factor is the friction after the elastic element, as this (usually 
complex) friction can hardly be compensated for by a controller. It appears from this scheme that 
the actual load influences the force tracking performance, as the load displacement (xi oa d) is an 
input to the system. With this general model, parts can be modelled as complex as seems 
necessary, or as complex as availability of system parameters allows for. All parameters should 
of course be transformed to the output axis domain, to fit in the general model. 
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Fig. 4. General model of a compliant actuator, (fitting in the shaded area of Fig. 3), consisting of a 
controlled (with controller Qn t ) internal position source (Hint) a compliant element with (variable) 
stiffness (K s ) and a (variable) friction model (Hfoct). These subsystems are possibly non-linear. The 
force over the spring element F s is the actually controlled force, because it is intrinsically measured. 
In case the real output force F act is measured, this measurement should be used for control, so that 
also the friction can be compensated. Position variables concerning the elastic element are: Xmt 
position of the internal side, xi oa d is the load position, x s the spring length. F re f is the desired 
reference force, F^ct the (external) friction force, whose sign depends on the motion direction. 

For every kind of compliant actuator the Hint forward path transfer function (Hf p ) has to be 
determined. For practical reasons (so that the transfer can be directly measured) the stiffness 
is included in the transfer. The load impedance is taken infinite corresponding to a fixed 
load. The Hf r i Ct is not included as the linear damping in this transfer function can not be 
measured since the load is fixed: 



Hf>{<D,K u y- 



-H mX (cD)K & 



(1) 



By applying feedback control the force tracking transfer function for fixed load conditions becomes: 



H fom {<o,K s )-- 



C mt H mt (a»K s 
l + C mt H mt ((o)K s 



(2) 
(3) 



The mechanical output impedance of this controlled actuator is: 

F -K 

Z(0),K s ) = ^^ = s - 

x load l + K s C mi H mi (co) 

This already shows that, if the Q nt Hint-gain falls off at higher frequencies, the impedance 
will become equal to -K s at those frequencies. It also shows that the mechanical output 
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impedance at lower frequencies can be modulated, as the actuator controller Q nt appears in 
the impedance transfer function. 

It should be noted that the Hi nt and the Hf r i Ct will in general not be linear. Saturation effects 
play an important role in determining the behaviour of elastic actuators. This means that for 
a prediction of actual performance in a certain operation conditions, Hi nt should be non 
linear and should contain the saturation characteristics. 



2.2 Model of the Pneumatic Cylinder Actuator 

For a pneumatic cylinder actuator (Fig. 5) the control problem is relatively complex, because 
of the fundamental strong nonlinearity in this class of actuators. In a system containing 
proportional valves, the control input is the valve current, which controls the valve flow 
area, which depending on actual up- and downstream pressures causes airflow. This airflow 
depends on the control input, the several flow-restrictions and the actual pressures, that 
depend on actual chamber volumes. The air flows determine the pressure changes in the 
two separate actuator chambers. The absolute pressure levels determine the stiffness and the 
pressure difference in the chambers determine the resulting force output. This results in a 
relation between the valve current and the chamber pressures that is highly non-linear with 
respect to the pressure and the chamber volumes. 
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Fig. 5. general outline of the components (left) and the model (right) of a typical pneumatic 
cylinder actuator (PCA). The force is indirectly controlled by controlling the actuator pressures 
corresponding to a control command. The double connecting lines are drawn to show that the 
system contains two valves to control the air supply to both cylinder chambers independently. 
The actuator is divided into a pressure build-up part, a force build-up part and an internal 
positon reconstruction part, to clarify the outline of the used model, p , £ and i are measured 
input parameters used in the inverse model. J c indicates the two valve control currents. 

When a proper model of the actuator system is available, it is possible to add non-linear 
blocks that mathematically invert the non-linearities of the pneumatic plant, resulting in an 
approximately linear system (Richer and Hurmuzlu 2000; Richer and Hurmuzlu 2000; Xiang 
and Wikander 2004), that is a system consisting of actual parts of the plant in combination 
with their approximate mathematical inverts. These inverts are also non-linear, and are a 
function of variables that have to be measured in real-time, like pressure and piston position. 
After this specific case of feedback linearization (Slotine and Li 1991) model based errors 
will be introduced that have to be compensated by the PID controller. By controlling the 
pressure levels independently in both chambers not only the force output but also the 
effective stiffness can be controlled. We used the model descriptions of Richer (Richer and 
Hurmuzlu 2000), including valves, tubing and cylinder chambers, and combined them with 
the block oriented linearization technique as described by Xiang (Xiang and Wikander 2004). 
It results in an approximately overall linear plant unless the actuator saturates. 
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2.3 Model of the Series Elastic Actuator 

The important advantage of a spring is that it allows treating the force control loop as a 
position control, because the spring length can be considered proportional to the force 
output. A higher compliance in the force sensor allows for higher control gains in the 
feedback spring length control loop. This way a better force control performance and 
actuator impact resistance can be achieved. Adverse effects of for example backlash and 
stick in transmissions can also be decreased this way. 

To fit a series elastic actuator into the general model (Fig. 4), the appropriate Hi nt and Hf r i Ct 
have to be obtained. Hint mainly depends on the dynamics of the motor used to drive the 
actuator, and can be calculated, in case of a brushless DC - driven SEA, with a basic linear 
motor model, shown in Fig. 6. This model of the SEA neglects all non-linearity, but this is 
less important as these will be largely compensated by the feedback of the spring length. 
This does not apply for saturation which will be considered separately. Saturation is an 
actuator inherent limitation that can not be compensated for. 
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Fig. 6. General outline of the components (left) and the model (right) of the Series Elastic 
Actuator, with added K s . Model of a DC-motor as torque source, including motor inertia 
and damping. k m is the motor constant, m m is the motor inertia, c m the motor damping, 
including friction components and back-emf of the motor. 

The K s is the physical spring-stiffness. This way all parameters in the H int transfer can be 
obtained from motor data. For the damping only the back-emf component will be known 
on beforehand; the damping component of system friction will have to be estimated. The 
output friction Hf r i Ct is difficult to obtain during the design phase. For now it will be 
neglected in the general model as it typically is small for a well designed series elastic 
actuator. 



3. Design of a Bowden-cable driven distributed series elastic actuator 

The basic idea, in designing this actuator was to detach the actual motor from the robot 
frame by the use of flexible Bowden cables (Fig. 7) to minimize the mass of the exoskeleton, 
which is important for an impedance controlled exoskeleton. 

The actuator system is constructed as a rotating joint, which has to function as a torque source. Such 
joints were integrated in the developed gait training robot (Fig. 1) as hip and knee joints. Both the 
flexion and the extension (bending and stretching motion) cable are a continuous unit (Fig. 7, sub 6). 
This was done for safety reasons. In case a cable breaks, its tension will be lost and no safety threat 
will occur to due unidirectional forces. This choice implies that the force transfer from the cables to 
the disc is friction based. The same applies to the cable connecting the springs; to prevent slipping, 
strips of high friction synthetic material have been fixed on the inside of the disk. Slippage would 
not affect the force output but could shorten the motion range. 
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The power transmission from motor to joint is realised by use of so called Bowden-cables. A 
Bowden-cable is a type of flexible cable used to transmit power by the movement of an inner cable 
relative to a hollow outer cable, generally a spiral steel wire with a plastic outer sheath, often 
containing an inner liner to reduce friction. Because Bowden-cables introduce orientation-, speed- 
and tension-dependent friction, friction compensation is needed. The angles of the curves in the cable 
and their radii appear to be the main determinants of the actual friction. Also wear and (pre-)tension of 
the cables are important factors. Because these parameters are hardly observable and their effects 
complexly interrelated, it is impossible to compensate the friction properly with feedforward 
control alone. Acceptable compensation may be achieved by introducing a feedback force control 
loop. This requires a force measurement located after the cable transmission. 
We chose to use springs for force measurement. A spring can be considered as a compliant 
and relatively low-cost force sensor, as its length can be considered proportional with the 
force. Two compression springs were connected to the actuator disk with a cable so that a 
torque spring is created in between the actuator disk and the lower segment. The two 
compression springs are pre-tensioned with the maximally desired force, so that the 
connecting cable will always be under tension during operation. 
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Fig. 7. Left a global lay-out of the proposed actuator system. The actuated joint can be lightweight 
as the motor is placed on the fixed world. On the right the final design of the joint alone is shown. 
The picture shows the actuated joint, the other side (not visible) of the Bowden cables is connected 
to a second disk driven by a servo motor; the latter side is situated on the fixed world. The cable 
with the two springs is the connection between the actuated disk and the joint output axis. 
Numbered parts: 1. sensor mounts, 2. joint end-stop, 3. upper connection side of the joint, 4. lower 
connection side of the joint, 5. set of two pretensioned compression springs, realizing a rotational 
spring around the main joint axis; the series elastic element of the actuator, 6. set of two pairs of 
uninterrupted Bowden cables, connected to the motor, transferring force to the actuator disk via 
friction. 7. actuator disc; this disc can rotate independently of both connection sides of the joint, or 
rather, it is connected to them via a force coupling, not an angle coupling. 

The concept is similar to Series Elastic Actuation (SEA), treated extensively in (Robinson 
2000). Its theoretical framework has to be only slightly adapted to be applicable on the 
actuator presented here. The differences with (Robinson 2000) are that we constructed a 
rotational joint instead of a linear, integrated the actuator with the robot-joint and added a 
Bowden cable transmission. Common cable drives have been used with SEA before, for 
example in the "Spring Turkey " (Pratt et al. 2001). 

The Bowden cable transmission might negatively influence the bandwidth of the actuator 
compared to common SEA, as it introduces friction and compliance into the position control loop. 
On the other hand, it is possible to select a heavier motor, as the motor weight is of no 
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importance in this setting and a larger motor inertia will increase the bandwidth of the actuator, 
as a smaller gear-ratio, thus a smaller reflected mass can be used (Robinson et al. 1999). 
Design parameters, besides the choice of motor and gearing, are the actuator disk diameter 
and the stiffness of the springs. The diameter is a compromise between the size of the joint 
and the low tension (thus friction) in the cables. The stiffness is a compromise of a high force 
control bandwidth versus minimal endpoint impedance combined. 

The proposed actuator was designed and built to function as a knee joint. With this set-up (Fig. 7), 
the measurements were carried out. It has an approximate peak torque output of 30 Nm. Four 
Bowden cables of 1.5m each were used. An LVDT sensor was used for spring length 
measurement. The model of the distributed SEA is in essence the same as that of the SEA (Fig. 6). 
The main difference is the much higher friction due to the cable transmission and the elasticity in 
the cable transmission itself. Details can be found elsewhere (Veneman et al. 2006). 

4. Performance of the actuators 

The developed models were mainly meant for design purposes. The performance of the 
actuator can be considered independently of such model assumptions. In this chapter 
several issues are considered that determine the feasibility of the actuator for the described 
application (compare (Hay ward and Astley 1996), (Morrell and Salisbury 1998)). 
Performance depends on many small constructive and sometimes controller decisions, so 
the outcome should be interpreted as a mere indication of the achievable performance with 
the presented type of actuation.. 
The performance will be considered in two consecutive issues: 

1. Bandwidth of force tracking with a fixed load: Up to what frequency can the output 
force of the actuator be modulated with feedback control? 

2. Reduction of the output-impedance: how well can the same controller decrease the 
mechanical output impedance of the actuator? 

For the measurements with the distributed SEA a Bowden cable course has to be defined, as 
this affects performance. The optimal course of a Bowden cable would be straight, as 
bending introduces friction and backlash. A realistic standard situation was defined in 
which the Bowden cables are bent over 90° with a radius of 0.8 m.. 

4.1 Bandwidth of force tracking with fixed load. 

To obtain acceptable force tracking feedback control is needed. The objectives of the 
feedback controller design were to improve the extent and accuracy of the force control, to 
reduce the apparent friction and inertia when back driven, and to reduce the sensitivity to 
load impedance variations. The force command was interpreted as a desired spring-length 
or pressure level, which then was controlled in a feedback loop, resulting in a torque 
command to the servomotor of the actuator or a current to the valves. 

A tuned PID controller is the most straightforward choice, and can be designed based on the 
open loop behaviour of the plant and/ or general tuning rules, like the ultimate cycle 
method of Ziegler and Nichols. This method has been used to tune feasible controllers. 
Due to the amount of noise in the spring length measurement (LVDT) of the SEA, the differential 
action of the controller introduced a lot of noise into the control command. To prevent this, the 
encoder measurements of the motor position were used in the differential part of the controller. 
The force tracking bandwidths were evaluated for force levels with increasing amplitude to 
investigate the effects of saturation. Force tracking transfer functions were identified using a 
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multisine signal with frequency content between 0.1 and 30Hz. The signal was crest 
optimized, to prevent appearance of high peaks in the composed signal. 

4.2 Reduction of the mechanical output-impedance 

In relation to the 'patient in charge 7 -mode it is important to determine the minimal 

mechanical output impedance. This is also called the back driveability of the device or 

actuator. The uncontrolled compliant actuators are already back driveable, but still have, 

depending on the actual configuration, high impedance. 

The output-impedance is measured by imposing a position trajectory upon the load position 

(xioad) and measuring the actuator force. For the pneumatic actuator and the series elastics 

actuator this external position perturbation was opposed by DC motor. For the distributed 

SEA the external position perturbation was applied manually. 

The hand was considered feasible as disturbance source, considered the intended 

application of the actuator. 



5. Results 

5.1 Pneumatic cylinder actuator 

Bandwidth of force tracking with fixed load 

The force-bandwidth of the PCA depends on the RMS of the amplitude of the reference input 
(Fig. 8) It appears that for low forces behaviour is predicted poorly by the model. This is 
probably due to the heavy influence of friction forces that were only basically modelled. 
Another possibility is that the measurement for small pressures is not accurate enough. For 
larger amplitudes the model appears to give at least a good indication of what the system can 
achieve. The reduction in the force tracking bandwidth for larger force levels are due to flow 
restrictions caused by the size of the inlet and outlet openings and the tube diameters and 
lengths. Larger inlet and outlet openings and tube diameters and shorter tubes reduce the flow 
restrictions and increase the force-tracking bandwidth for larger desired force levels. 



I nrrtiLKinr hn J» i Jtf- KA 



-mtfil' mi [Hotter Kfl, 




J l..-.„., J ,i..JL,l 



Fig. 8. Performance of the pneumatic cylinder actuator. Left: Force tracking bandwidth as function 
of reference RMS amplitude. Right: Measurement of controlled impedance. Bode plot of the force 
response to a l-5Hz multisine motion profile. For comparison the uncontrolled values are added, 
together with the case in which both valves are switched open to open air. 

Minimal mechanical output-impedance 

To show the limits of impedance reduction in the pneumatic cylinder, some controlled (with 

reference F= N) impedance responses are shown in Fig. 8. It appears that the impedance is 
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hardly reducible compared to the uncontrolled situation. The impedance response in case 
both cylinder chambers are open to the ambient is also shown. The impedance in this 
situation is caused by the friction of the system, together with the resistance to airflow in- 
/ out-lets. 



5.2 Series elastic actuator 

Bandwidth of force tracking with fixed load 

The closed loop force tracking depends on both the system transfer and on the implemented 
controller.As saturation plays a role already at very small RMS amplitudes of force, the 
actual force tracking is compared to force tracking in the model that included the saturation 
characteristics of the used motor. The transfer function of the non-linear model is 
determined in the same way as in the measurements by simulation with as input a force 
reference signal that was composed of multiple sinuses. The amplitude is given as RMS 
value of the multisine reference input. 
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Fig. 9. Performance of the series elastic actuator. Left: Force tracking bandwidth as 
function of reference RMS amplitude. Right: gain of the controlled impedance transfer. 
For high frequencies the impedance becomes the stiffness of the springs, which is the 
intrinsic impedance. For low frequencies the model underestimates the 
impedance/ stiffness. This is caused by the coulomb friction component, which gives the 
force response a lower limit. 

We obtained a good agreement between model and measurements (Fig. 9). Similar as in the 
PC A saturation limits the bandwidth at higher desired force levels. From the same figure it 
can also be seen that the bandwidth demands, 12 Hz for small forces (below 100 N) and 4 
Hz for large forces, are met 
Minimal mechanical output-impedance 

The controlled impedance depends on both system and controller. Here an example is given 
(using the PID same controller with reference value F= N), to demonstrate some typical 
properties. In general the measurements were well predicted, except for low frequencies (<2 
Hz). Where the model predicts a further decrease in impedance, the measurement indicates 
a lower boundary at about 5500 N/m. Further measurements showed that this was actually 
a lower boundary on the force output, which is caused by the coulomb friction component. 
Combined with certain motion amplitude, this results in the measured impedance boundary. 
For high frequencies the impedance approaches the intrinsic stiffness (K s ) that is the spring 
stiffness. 
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5.3 Distributed series elastic actuator 

Bandwidth of force tracking with fixed load 

The performance of the feedback controlled actuator with the implemented PIT) controller is 

presented in Fig. 10. Again the RMS amplitude of the input was varied. 

It can be seen that the controlled bandwidth of the actuator is over 20 Hz for the measured 

range of torques in a normal Bowden cable course (Fig. 10 left). For smaller than 1 Nm 

torques the transfer falls off already at a lower frequency (not shown). 

In case the cable is excessively bent, the effects of friction increase dramatically and cause a 

dramatic decrease of performance (Fig. 10 right). 
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Fig. 10. Force tracking bandwidth of the distributed series elastic actuator. The RMS 
amplitude of the reference was varied. Left: the orientation was 90° with a bending radius of 
0.8m. Right: the orientation was 180° with a bending radius of 0.4m. 

Minimal mechanical output-impedance 

The reduction of output impedance as achieved by using the PID controller with a zero 
reference force is shown in Fig. 11. From this Figure it can be concluded that a reduction of 
10 - 13 dB (factor 3 - 4.6) of the mechanical impedance can be realized with feedback control 
in the frequency range of application. The values of the torques range from negligible up to 
0.7 Nm for around 4 Hz motion. The noticeable effects of cable stick forces on the output 
force are also reduced. The time domain plot (Fig. 11 right) shows the small peaks caused by 
static friction compared to the overall torque response. Tests with a walking subject 
connected to this joint confirmed that the controlled impedance was low enough to 
experience unhindered lower leg motion. The minimal impedance was much smaller 
compared to the SEA since there was much less friction after the spring. 



6. Discussion 

6.1 modelling both SEA and PCA as elastic actuators 

It was possible to model a series elastic actuator and a pneumatic cylinder actuator within 
one general framework of compliant actuation. This general model, as presented in Fig. 4., 
clearly shows the effect of the intrinsic stiffness (K s ) and controller design on the force 
tracking bandwidth, and on the controlled and uncontrolled mechanical output impedance. 
For the comparison of compliant actuators the intrinsic stiffness (K s ) of each specific actuator 
has to be defined, representing the average stiffness of the actuator during operation. In case 
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of the SEA K s is the stiffness of the elastic element. In case of the PCA K s was defined as the 
stiffness of the cylinder around its equilibrium position, at the pressure halfway in between 
ambient and buffer pressure, which will be about the average stiffness during operation. We 
found that saturation behaviour and output friction are very important in modelling elastic 
actuators as they greatly affect performance 
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Fig. 11. Impedance of distributed SEA joint with compliance spring. Left: Bode plot of the 
impedance identifications of several measurement-series in both controlled and 
uncontrolled situations. The input motion disturbance signal was applied by hand. Only the 
frequency range with appropriate coherence is shown. Right: Typical torque response of the 
actuator to external motion, while controlled to zero force output. The small peaks are 
caused by stick, appearing when the motion is reversed. 



6.2 Similarities and differences between SEA and PCA 

Depending on its dimensions a PCA can be considered as a realisation of a 'compliant actuator 7 . 
Since PCA needs feedback linearization to be controlled properly, it appeared that it was much 
harder to predict and control a PCA than a SEA. The feedback linearization needs a precise 
actuator model, which in general will not be available beforehand. Important parameters, 
temperatures, actual buffer pressure et cetera, were left out of the modelling and will demand 
extra sensors if actual temperatures and the buffer pressure will be included in the feedback 
linearization. The PCA saturation behaviour was more complex than for the SEA. Therefore the 
design of a PCA demanded much more fine-tuning. The design and prediction of the 
performance of the SEA can be based on available data-sheet parameters. 

The spring stiffness of a linear SEA with continuous force output of 500 - 1000N appeared to be 
in the order of 50-300 kN/m, resulting in a large force bandwidth of 4-10 Hz. The stiffness of the 
elastic element in our design was 54.5 kN/m. In a PCA the intrinsic stiffness is not constant; the 
range is defined by cylinder dimensions and the actual pressures and stroke position. The range 
of intrinsic stiffness of the selected cylinder during operation lies in between about 6 and 40 
kN/m, depending on the feasible working range (Fig. 12). The effective stiffness as defined 
before is 11.3 kN/m. The effective stiffness of a PCA is on average roughly 20% of the designed 
linear SEA. Choosing the same low stiffness for the elastic element in a SEA would affect the 
large force bandwidth too much. The minimal stiffness/ impedance of a SEA in the controlled 
situation is typically lower than the achievable stiffness/ impedance of a PCA. When a PCA is 
controlled based on pressure measurements its minimal impedance is determined by both the 



144 



Mobile Robots, Towards New Applications 



piston friction and the airflow resistance of the air in- and outlets. These properties depend on the 
cylinder and might be improvable by redesign, but can hardly be compensated by controller 
design. In the PCA, flow restrictions (relative small flow areas in cylinder, tubes) appeared to 
hinder a low-impedance control, i.e. the out-flow of the cylinder was mainly determined by these 
restrictions and not by valve control limitations. It may be possible to re-design a PCA for to 
minimize the mechanical output impedance. This might however affect system safety, as these 
small outlets serve as speed limiters. Since the friction is difficult to predict proper impedance 
control can only be achieved by adding a force sensor to the PCA. Another option would be 
designing pneumatic cylinders with a very low friction. These are commercially available in 
small size pneumatics, but to our knowledge not in the sizes needed for gait rehabilitation. 
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Fig. 12. Comparison between the stiffness of the series elastic actuator and the pneumatic 
cylinder stiffness. 

Both the SEA and PCA can be designed to achieve a comparable force tracking bandwidth for 
'midrange' force amplitudes. For smaller amplitudes the SEA outperforms the PCA in the 
achievable bandwidth and in accuracy, mainly due the smaller output friction. The force 
bandwidth for low amplitudes for the SEA was 30Hz and for the PCA about 25Hz. However, the 
SEA force bandwidth decreased faster with increasing amplitude (compare Fig. 8 and Fig. 9) 
Both actuators appeared to be impact proof due their intrinsic compliant behaviour at higher 
frequencies. The SEA appeared to be easier and better controllable in an impedance range at low 
frequencies, due less output friction, and due a more accurate force measuring. 



6.3 Actuator performance of the distributed SEA 

Using a basic PID controller the distributed SEA was able to achieve a force tracking bandwidth 
up to a bandwidth of 20 Hz and an expected large force bandwidth of up to 11 Hz. For small 
torque amplitudes and heavy Bowden cable bending this bandwidth decreased to as low as 2-3 
Hz. This could to some extent be compensated for by increasing controller gain, which is possible 
as long no saturation for the motor is reached. Doing this in practise would demand a good 
observation of the bending radius of the cable since a return to the standard cable course or 
normal amplitudes could result easily in instability due to the decrease of physical friction. These 
problems can partly be avoided by constructing the robot in such a way that cables are never bent 
excessively. The decrease in bandwidth for (very) small torques is supposed to be an inherent 
limitation of the system and depends on the needed (pre-) tension in the cables and the bending 
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radius. Stick-slip compensating control strategies like dither signals, seem to be poorly applicable 
in Bowden cables. A feasible possibility could be a direct measurement of the friction forces, and 
including this measurement in the feedback controller. 

The mechanical output impedance was shown to be considerably reducible by 10-13dB by 
feedback control compared to the impedance without feedback control. For our application a 
maximum torque of 0.7 Nm during 4 Hz imposed motion is acceptable. For gait rehabilitation 
practice such frequencies are high, and such torques are negligible. From the time domain plot it 
appeared that at every reverse of the velocity small torque peaks occur. Their size of about 0.1 Nm 
is however again negligible and acceptable for the application in a gait trainer robot. 
Friction or stick was the most apparent non-linearity in the system. It was also shown that 
stick hardly distorted the controlled force output at frequencies below 15 Hz. 
The spring stiffness affects actuator performance. We found that that a system with a stiffer 
spring has a higher bandwidth in open loop mode and is relatively less damped than with 
more compliant springs. When using stiffer spring we saw more non-linear distortions. This 
might be explained by the smaller motions of the motor that negatively affect the linearity of 
the relation between control command and motor torque. Spring deflections will be smaller 
when stiffer springs are used. As a result the non linear effects of backlash in the 
transmissions will then become more manifest. 

Spring stiffness variations showed that both a too stiff and a too compliant spring can 
worsen performance. A stiff spring reduces the maximum allowable controller gain. The 
relatively low control gain then causes a larger effect of stick in the force output, resulting 
in a less smooth output in general. Low spring stiffness, on the other side, decreases the 
bandwidth of the system as demanded displacements, velocities and accelerations 
increase, which will saturate at a certain level. For every specific application this trade-off 
has to be made. In the design phase the required bandwidth or the amount and 
implication of stick might not be known beforehand. However springs can easily be 
exchanged. 

The presence of the compliant element allows for a much higher position control gain, 
implying a better internal error rejection (Robinson 2000). Furthermore stability can be 
assured if the controller is tuned to the realistic cable situation with the least friction, so that 
changing the cable situation will increase the friction, which will lower performance but not 
result in instability. Friction forces always stabilize a system and the possible source of 
instability can only be overcompensation. This can be concluded considering a Lyapunov- 
stable linear system; if extra non-linear friction is added, more energy will be dissipated 
from the system, so that it will stay Lyapunov-stable. 

Improvements of the actuator system could be realized by reducing system complexity, size 
and weight and by controller improvements. The complexity of the system can be slightly 
reduced by using two (thicker) Bowden cables instead of four. This has been tested and 
appeared to be equivalent in performance. Also the LVDT sensor has been replaced by a 
linear slider potentiometer, which is both smaller and cheaper. Possibly a torsion-spring 
element could be constructed instead of the two compression springs, which would reduce 
size and weight. Feasible commercial torsion springs however were not found. The 
controller outline could be changed to deal better with the specific non-linearities, like the so 
called Tate motor processing 7 as described in (Pratt et al. 2004), or by using voltage/ velocity 
control instead of current/ torque control. Finally, the joint could be redesigned to optimize 
the cable replacement procedure, and durability tests should be done to determine the 
needed frequency of cable replacement. 
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7. Conclusions 

The designed SEA was slightly superior to the PCA for the described application in a gait 
rehabilitation robot, based on performance metrics. Other disadvantages of the PCA are the 
typical noise, dependency on pressurized air facilities and the complexity of the needed 
controller, which affects the robustness of the system. The SEA and PCA we evaluated were 
dimensioned to actuate a gait rehabilitation robot. From the evaluation of both type of 
actuators it can be concluded that both can be designed to meet power and bandwidth 
requirements, resulting in actuators of comparable weight and size. 

However, due the high weight neither one system can be considered truly optimal for use in 
exoskeleton-type rehabilitation robot. To minimize the weight we designed and evaluated a 
new Bowden-cable-based series elastic actuation system. A robot joint containing this 
actuation system was constructed. It was shown that the performance demands of realizing 
a safe, lightweight, adjustable and powerful torque source were met. A force bandwidth of 
up to 20 Hz appeared feasible for small forces, when extreme cable bending was avoided. 
Lower bandwidths were found for higher forces, as a consequence of saturation ranging 
from 6-10 Hz. This meets the requirements we derived for the control of human gait using 
an impedance controlled exoskeleton. Distributed series elastic actuation appeared also 
suitable for low impedance control to facilitate unhindered motion, with maximal torque 
peaks of 0.7 Nm at 4 Hz motions on a scale of a 30 Nm actuator (less than 2.5%). This implies 
that a robot using these joints would be able to be used in both a high impedance 'robot-in- 
charge' as well as in a low impedance 'patient in charge 7 task for rehabilitation robots, as far 
as joint and actuator performance are considered. 

Beside this typical application in the LOPES project, the principle of actuation could find 
wider use in all kinds of exoskeleton-like kinaesthetic interfaces. Of course dimensions 
should be optimized for every specific application. 

The developed distributed series elastic actuator system showed that the principle of Series Elastic 
Actuation can be applied using a heavy friction transmission like Bowden cables. Despite the 
power loss and the introduced heavy friction, the actuator still functions well as Series Elastic 
Actuator. Heavier motors can be selected since they are detached from the actual robot 
construction. The friction of Bowden cables can well be compensated with feedback control. 
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1. Introduction 

Robots have been successfully employed in industrial settings to improve productivity and 
perform dangerous or monotonous tasks. More recently, attention has turned to the use of 
robots to aid humans outside the industrial environment, in places such as the home or 
office. For example, as the population in the developed world ages, robots that can interact 
with humans in a safe and friendly manner while performing necessary home-care/ daily 
living tasks would allow more seniors to maintain their independence. Such devices could 
alleviate some of the non-medical workload from health-care professionals, and reduce 
growing healthcare costs. To achieve such objectives, robotic devices must become more safe 
and user friendly. Untrained users need to feel comfortable and confident when interacting 
with a device that, unlike most passive household appliances, is potentially active in its 
interaction with the user. 

Two key issues hampering the entry of robots into unstructured environments 
populated by humans are safety and dependability (Corke, 1999; Lee, Bien et al., 2001). 
To ensure the safety and intuitiveness of the interaction, the complete system must 
incorporate (i) safe mechanical design, (ii) human friendly interfaces such as natural 
language interaction and (iii) safe planning and control strategies. Our work focuses 
on this third item. The design of safe planning and control strategies can be divided 
into three key components: safe planning, human interaction monitoring, and safe 
control (Kulic and Croft, 2003). Discussion of the human monitoring and control 
system components can be found in Kulic and Croft (Kulic, 2005; Kulic and Croft, 
2006a; Kulic and Croft, 2006b). Here, we specifically address the planning aspects 
related to safety in human-robot interaction (Kulic and Croft, 2005). First we discuss 
the development of a motion planning approach for robots that minimize a danger 
index based on the physical parameters of the robot and its proximity to the user. Then 
we present experiments where users are asked to evaluate the robot motions with 
regard to their perception of the safety of the robot motion. Subjects reported 
significantly less anxiety and surprise, and reported feeling more calm when safe 
planned motions were presented, as compared to a conventional motion planner. 



1 This work is revised and expanded from work reported in Kulic and Croft (Kulic, 2005; Kulic and 
Croft, 2006a; Kulic and Croft, 2006b). 
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1.1 Related Work 

In industrial applications, the safety of human-robot interaction is effected by isolating the 
robot from the human (Gaskill and Went, 1996; Corke, 1999; RIA/ANSI, 1999). In effect 
there is no interaction. As robots move from isolated industrial environments to interactive 
environments, this approach is no longer tenable (Corke, 1999). Three main approaches can 
be used to mitigate the risk during human-robot interaction: (i) redesign the system to 
eliminate the hazard, (ii) control the hazard through electronic or physical safeguards, and, 
(iii) warn the operator/ user, either during operation or by training (RIA/ANSI, 1999). While 
the warn/ train option has been used in industry, it had not been deemed effective in that 
setting (RIA/ANSI, 1999), and is even less suitable for robot interaction with untrained 
users. Examples of redesign include using a whole-body robot visco-elastic covering, and 
the use of spherical and compliant joints (Yamada, Hirawawa et al., 1997; Yamada, 
YamamotoetaL, 1999). 

In unstructured environments, mechanical design alone is not adequate to ensure safe and 
human friendly interaction. Additional safety measures, utilizing system control and planning, 
are necessary. Several approaches have been proposed for ensuring safety through control. 
They focus on either slowing down or stopping when a hazardous situation is identified 
(Bearveldt, 1993; Yamada, Hirawawa et al., 1997; Zurada, Wright et al., 2001), moving to evade 
contact (Traver, del Pobil et al., 2000), or trying to minimize the impact force if contact occurs 
(Lew, Jou et al., 2000). A key problem for all of these control methods is to identify when safety 
is threatened. One approach is to use tactile sensors and force/ torque sensors to identify a 
hazard when unplanned contact occurs (Yamada, Hirawawa et al., 1997). Recently, Ikuta et al. 
(Ikuta and Nokata, 2003) developed a danger evaluation method using the potential impact 
force as an evaluation measure. In their work, the danger index is defined as a product of 
factors which affect the potential impact force between the robot and the human, such as 
relative distance, relative velocity, robot inertia and robot stiffness. 

Motion planning and the a priori identification of potentially hazardous situations as a means of 
reducing potential robot-safety hazards has received less attention than control-based (reactive) 
techniques. However, safe planning is important for any interaction that involves motion in a 
human environment, especially those that may contain additional obstacles. Application examples 
include service scenarios such as a dish clearing robot (Bearveldt, 1993), services for the disabled, 
such as approaching the human for a feeding task (Kawamura, Bagchi et al., 1995; Guglielmelli, 
Dario et al., 1996), and pick and place tasks for picking up and delivering common objects 
(Bischoff and Graefe, 2004). Including safety criteria at the planning stage can place the robot in a 
better position to respond to unanticipated safety events. Planning is thus used to improve the 
control outcome, similar to using smooth trajectory design to improve tracking (Erkorkmaz and 
Altintas, 2001; Macfarlane and Croft, 2003). 

Several authors consider an a priori evaluation of the workspace to determine motion 
parameters within the various zones of the workspace (Bearveldt, 1993; Yamada, Hirawawa 
et al., 1997). Blanco et al. (Blanco, Balaguer et al., 2002) use distance measures from a laser 
scanner to generate a Voronoi diagram of the workspace of a mobile manipulator 
performing co-operative load carrying with a human. Since the Voronoi diagram maximizes 
distance from obstacles, paths generated along the Voronoi diagram present the safest 
course in terms of collision potential. 

Khatib (Khatib, 1986) developed the potential field approach. In this method, the environment is 
described by an attractive (goal) potential field, which acts on the end effector, and a repulsive 
(obstacle) potential field, which acts on the entire robot body. The potential field is specified in the 
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operational space. The potential field is used to generate forces to pull the robot away from any 
obstacles, and the end effector towards the goal. This approach does not require extensive pre- 
computation of the global path, can operate on-line, and can be easily adapted to sensor based 
planning and dynamic obstacles. When a redundant robot is used, this approach can be extended 
to allow the robot to continue executing the task while avoiding obstacles (Khatib, 1995). 
Maciejewski and Klein (Maciejewski and Klein, 1985) proposed a similar method for redundant 
manipulators and tasks where a goal trajectory is specified, and not just a goal location. In this 
approach, the force generated by the obstacle avoidance potential field is mapped to the null space 
of the redundant manipulator, so that the robot can continue to execute the goal trajectory while 
using its redundant degrees of freedom to avoid obstacles. A major issue with these planning 
methods is that only local search is used, so the robot can reach a local-minimum that is not at the 
goal location. A second issue is the formulation of the forces applied to the robot in the operational 
space. This requires the use of the robot Jacobian to translate these forces to joint torques, and 
introduces position and velocity error near any robot singularities. 

Nokata et al. (Nokata, Ikuta et al., 2002) use a danger index based on the impact force 
between a human and the end effector. The danger index is the ratio of the actual force to 
the largest "safe" impact force (an impact force that does not cause injury to the human). 
The danger index is calculated based on factors such as the distance and velocity between 
the human and the manipulator end effector. Two approaches are proposed for planning the 
motion of a planar robot end effector: minimizing the greatest danger index along the path, 
and minimizing the total amount (integral) of the danger index along the path. However, 
their approach considers only the end effector motion. 

Chen and Zalzala (Chen and Zalzala, 1997) use the distance between the robot and any 
obstacles as a measure of "safeness" in the cost function for path planning for mobile 
manipulators. A genetic programming approach is used to generate the optimum path 
given multiple optimization criteria, including actuator torque minimization, torque 
distribution between joints, obstacle avoidance and manipulability. 

Oustaloup et al. (Oustaloup, Orsoni et al., 2003) describe a method for path planning using 
potential fields. The method is described for mobile robots, but is extendable to robot 
manipulators in configuration space. In pre-planning, obstacles are classified according to 
how much danger they pose. The magnitude of the potential field gradient is varied by 
fractional integration, with a steeper slope for more dangerous obstacles. The fractional 
differentiation approach allows for a smooth transition between obstacles, however, this 
approach is susceptible to local minima. 

Brock and Khatib (Brock and Khatib, 2002) describe the Elastic Strips framework for motion 
planning for highly articulated robots moving in a human environment. This method assumes a 
rough plan for accomplishing the task is available, and is fine-tuned on-line based on changes in 
the environment. The potential field method in operational space is used to plan the motion, with 
an attractive field pulling the robot towards the nominal off-line plan, and a repulsive force 
pushing the robot away from any obstacles. The existence of the pre-planned global path to the 
goal ensures that the robot does not get stuck in local minima. For redundant manipulators, an 
additional posture potential field is defined to specify a preferred posture for the robot. The 
posture field is projected into the null-space of the manipulator, so that it does not interfere with 
task execution. Although their paper does not deal explicitly with ensuring safety, the posture 
potential can be used to formulate safety-based constraints. 

Most path planning algorithms for human environments focus on maximizing the distance 
between the robot and any obstacles in the environment. In this work it is proposed that the 
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robot posture can also be optimized during path planning to significantly improve the 
safety of the manipulator. 



1 .2 System Overview 

The system architecture assumes a user-directed robot system. The user must initiate each 
interaction, but the robot has sufficient autonomy to perform commanded actions without 
detailed instructions from the user. An overview of the idealized system is presented in Fig. 
1. The user issues a command to the robot to initiate the interaction. The command 
interpreter translates the natural language command (e.g.: pick up the red cup) into a set of 
target locations and actions (e.g., execute a grip maneuver at coordinates [x,y,z]). The 
planning module is divided into two parts: the global path planner and the local trajectory 
planner. The global planner module begins planning a geometric path for the robot over 
large segments of the task, utilizing the safety strategy described herein. Segment end points 
are defined by locations where the robot must stop and execute a grip or release maneuver. 
For example, one path segment is defined from the initial position of the robot to the object 
to be picked up. The local planner generates the trajectory along the globally planned path 
based on real-time information obtained during task execution. The local planner generates 
the required control signal at each control point. Because the local planner utilizes real-time 
information, it generates the trajectory in short segments. 

During the interaction, the user is monitored to assess the user's level of approval of robot 
actions. The local planner uses this information to modify the velocity of the robot along the 
planned path. The safety control module evaluates the safety of the plan generated by the 
trajectory planner at each control step. The safety control module initiates a deviation from 
the planned path if a change in the environment is detected that threatens the safety of the 
interaction. This deviation will move the robot to a safer location. Meanwhile, the recovery 
evaluator will initiate a re-assessment of the plan and initiate re-planning if necessary. 
The focus of this work is the development of a motion planning strategy that explicitly 
defines a measure of danger in the interaction, and incorporates this measure as a criterion 
during planning. The developed planning strategy is a standalone component that can be 
used as part of the architecture described above, or as part of a different control and 
planning architecture. In this work, we develop the danger evaluation measure and its use 
during path planning. Following this description, we present simulations and experiments 
to demonstrate the behavior and effectiveness of the planner, and characterize the user 
evaluation of the proposed strategy. 





User Monitoring and 
Intent Estimation 



Fig. 1. System Overview. 
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2. Measuring the safety of a planned interaction 

A hazard requiring a change in robot behavior can be defined by a minimum distance between 
the robot and the person (Bearveldt, 1993; Traver, del Pobil et al., 2000), or by using a threshold 
level of the danger index based on impact force (Nokata, Ikuta et al., 2002; Ikuta and Nokata, 
2003). Here, an index similar to (Nokata, Ikuta et al., 2002; Ikuta and Nokata, 2003) is proposed, 
and applied to configuration space planning of the robot motion. By selecting safer 
configurations at the planning stage, potential hazards can be avoided, and the computational 
load for hazard response during real time control can be reduced, as shown in Fig. 2. In both 
panels the robot has the same end-effector location, but in the panel of the right (b), the hazard to 
the user is minimized by the posture adopted by the robot. 

Safe planning is an important component of the safety strategy. For example, if the path to be 
followed is planned with a general path planning method, the robot may spend the majority of 
the path in high inertia configurations. If the user suddenly moves closer to the robot, the 
potential collision impact force will be much higher than if the robot had been in a low inertia 
configuration, regardless of the real-time controller used to deal with potential collision events. 
When selecting a path planning strategy, there is a tradeoff between fast local methods that may 
fail to find the goal, and slow global methods (Latombe, 1991). To exploit advantages of both 
methods, recent path planning algorithms have used a hybrid approach, where global path 
planning is used to find a coarse region through which the robot should pass, and local 
methods are used to find the exact path through the region (Brock and Khatib, 2002). 
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Fig. 2. Planning a safe interaction. Posture (b) has minimized potential hazard to the user. 

Similarly, in this approach, the global planner generates a safe contiguous region in space 
through which the robot can move to complete the given task. This region in space is 
described by a set of contiguous configurations, which represent the path. It is then left to 
the on-line trajectory planner to generate the exact path in the region, and the trajectory 
along that path. This trajectory is evaluated and, if necessary, corrected at every control step 
by the safety module to handle the real-time aspects of the interaction. 

Since the task planning is done following a user request, the global planner must execute 
within several seconds at most, to avoid a significant delay between a user request and 
robot response. To ease the computational load on the global planner, the task is separated 
into segments. Natural segment separation points occur when the robot is required to pause 
at a particular location, for example at each grasp or release point. Only the first segment 
must planned before the planned path can be passed on to the local planner and the robot 
can begin executing the task. In this way, global planning of the next segment can continue 
in parallel with execution of the current segment. 
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2.1 Danger Criterion 

The planning module uses the best-first planning approach (Latombe, 1991). In this method, the 
robot configuration space is discretized into 0.1 rad cells, and a path is found by iteratively 
building a tree of configurations, with the first configuration at the root. At each iteration step, the 
neighbours of the leaf configuration with the lowest cost value are added to the tree. The 
algorithm therefore follows the steepest descent of the cost function, and escapes from local 
minima by well-filling. The search stops when the target configuration is reached or the entire 
space has been mapped. The algorithm is resolution complete. In cases when the number of 
degrees of freedom (DoF) of the robot affecting gross end-effector motion are small (less than 5), 
the best-first planning approach provides a fast and reliable solution (Latombe, 1991). For highly 
redundant robots, a different search strategy can be employed, such as randomized planning 
(Barraquand and Latombe, 1991), or probabilistic roadmap planning (Kavraki, Svestka et al., 1996; 
Ahuactzin and Gupta, 1999; Choset and Burdick, 2000; Yu and Gupta, 2000). For example, either 
random sampling of the configuration space (Kavraki, Svestka et al., 1996), or the Generalized 
Voronoi Graph (Choset and Burdick, 2000), can be used to generate a roadmap of the connected 
free regions in the configuration space. The roadmap represents a subspace of the entire 
configuration space. The search based on lowering the danger criterion can then be applied to the 
roadmap, rather than the entire configuration space, reducing the search time for high DoF 
manipulators. However, the search criteria presented herein remain identical regardless of the 
search strategy used. The safest path is found by searching for contiguous regions that: (i) remain 
free of obstacles, (ii) lead to the goal, and, (iii) minimize a measure of danger (a danger criterion). 
The planning algorithm seeks a path that minimizes a cost function consisting of a quadratic goal 
seeking function, a quadratic obstacle avoidance function, and the danger criterion (DC). 
The danger criterion is the central contribution of the planner cost function. Since path 
planning (as opposed to trajectory planning) does not consider robot velocities, a 
configuration-based (quasi-static) danger criterion is required. To be effective, the danger 
criterion should be constructed from measures that contribute to reducing the impact force 
in the case of unexpected human-robot impact, as well as reducing of the likelihood of 
impact. These can include the relative distance between the robot and the user, the robot 
stiffness, the robot inertia, the end-effector movement between contiguous configurations, 
or some combination of these measures, similar to those proposed in (Ikuta and Nokata, 
2003). In (Nokata, Ikuta et al., 2002), Nokata et al. use the danger index to find an optimum 
safe path, however, only the end-effector trajectory with respect to the user is considered. 
As an expansion of this approach, a safe path for the entire robot structure is planned, 
explicitly planning the robot posture. However, since some of the factors affecting danger 
can conflict (e.g., a low stiffness configuration can also be high inertia configuration) it is 
important to re-formulate the danger criterion so that conflicting factors do not act to 
cancel each other out. Herein, the robot inertia and the relative distance between the robot 
and the user's center of mass are used. The robot stiffness was not included as it can be 
more effectively lowered through mechanical design (Bicchi, Rizzini et al., 2001; Ikuta and 
Nokata, 2003). Instead, the proposed approach modifies the robot inertia, lowering the 
effective impedance of the robot. Dynamic factors, such as the relative velocity and 
acceleration between the robot and the user, are handled by the trajectory planner and the 
safety module (Kulic and Croft, 2006b). 

For optimization purposes, a scalar value representing the effective robot inertia at each 
configuration must be computed. For a general robot architecture, where the robot's 
inertia may be distributed in more than one plane, the largest eigenvalue of the 3x3 
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inertia tensor may be used as the scalar measure. For robots with a single sagittal2 plane 
(e.g. anthropomorphic, SCARA), the scalar inertial value is extracted by calculating the 
robot inertia about an axis originating at the robot base and normal to the robot's 
sagittal plane. 

I s = v T I b v • (1) 

Here, I s is the inertia about the v axis, v is the unit vector normal to the robot sagittal plane 
and h is the 3x3 robot inertia tensor about the base. 

For each danger criterion factor, a potential field function is formulated as a quadratic function. 
Quadratic potential functions are most commonly used in general potential field planners. They 
have good stabilization characteristics, since the gradient converges linearly towards zero as the 
robot's configuration approaches the minimum (Khatib, 1986; Latombe, 1991). 

2.1.1 Sum-Based Criterion 

Two danger criterion formulations are proposed: a sum-based and a product-based 
criterion. For the sum-based danger criterion, the inertia factor is: 

■sum b m 

where, m is the total mass of the robot. This function can be interpreted as a quadratic 
attractive function, attracting each link towards the robot base. 

The relative distance factor for the sum-based danger criterion is implemented by a 
repulsive function between the user and the robot center of mass (CoM). The center of mass 
distance is used (instead of the closest point distance) to allow the robot end-effector to 
contact the user during interaction tasks, while maximizing the distance between the user 
and the bulk of the robot. The potential field is described by equation (3) below: 



CMO -| D CM " D min| • ( 3 ) 
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In Equation (3), Dcm is the current distance between the robot center of mass and the user, 
Dmin is the minimum allowable distance between the robot centre of mass and the user, and 
Dcmo is computed as the difference between these two distance measures (namely, the 
separation distance above the minimum limit). D max is the distance at which Dcmo, the 
current distance above the minimum limit, no longer contributes to the cost function (for 
example, if no human is visible in the environment). £ is a small number used to limit the 
function for Dcm near D m ; M . This potential field is analogous to an obstacle potential field 
acting between the centers of mass of the user and the robot. 

The sum based danger criterion is comprised of the inertia factor (Equation (2)) and the 
centre of mass distance factor described above (Equation (3)), as follows: 

DCsum=Wi.f |sum (ls) + W d .f CMsum (D CM ) ( 4 ) 

Here, W* and Wd are weights of the inertia and distance term, scaled such that 
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Wi + Wd = 1. The weights Wj and Wd are tuned based on the particular robot structure. For low 
inertia robots, and when the robot is close to the user, the distance factor will dominate the danger 
criterion, because the distance factor approaches infinity as the robot approaches the person. If 
inertia reducing behavior is desired for the path in these cases, Wi should be greater than Wd. 

2.1 .2 Product Based Criterion 

For the product-based danger criterion, the criteria are scaled such that for each potential 
function, the level of danger is indicated within the range [0 - 1]. Values greater than one 
indicate an unsafe configuration. 
The product based inertia criterion is defined as: 

f, (IS)=T^-. P) 

prod 'max 

where, I max is the maximum safe value of the robot inertia. In the simulations described in 
Section 0, the maximum robot inertia is used; however, a lower value can be used for high- 
inertia manipulators. In this case, the maximum safe value can be established based on the 
largest force magnitude that does not cause pain (Yamada, Suita et al., 1996) and the 
maximum robot acceleration. 

For the product based distance criterion, similar to the sum based distance criterion, the 
center of mass distance between the robot and the user is used. The relative distance 
criterion for the product-based danger criterion is: 

, l 2 ( 6 ) 
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The scaling constant k is used to scale the potential function such that the value of the 
potential function is zero when the distance between the user and the robot is large enough 
(larger than D max ), and is one when the distance between the user and the robot is the 
minimum allowable distance (D m ; n ): 

^2 (7) 



k = 
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Values of the product-based distance criterion above one indicate an unsafe distance. 
The product-based danger criterion is then computed as a product of these contributing factors. 

DC Prod^l prod ('s)-f CMprod ( D CM) (8) 

2.1.3 Goal and Obstacle Potential Fields 

For the goal seeking and obstacle avoidance functions, the customary quadratic potential 
field functions are used (Khatib, 1986; Latombe, 1991; Brock and Khatib, 2002). The goal 
seeking function fc is defined as: 

f G (D G ) = 7 D G 2 ' (9) 

where, Dg is the distance between the end-effector and the goal. 
The obstacle avoidance function fa is defined as: 
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where, Do is the distance between the robot and the nearest obstacle, Domin is the distance 
from the obstacle at which the obstacle begins to repel the robot (the influence distance). For 
the obstacle avoidance function, the distance between the robot and the nearest obstacle is 
taken as the distance between the closest point on the robot and the closest point on the 
obstacle. The distance between the robot and the nearest obstacle, as well as the distance 
between the robot and the non-interacting parts of the user are estimated using the 
hierarchy of spheres representation (Martinez-Salvador, del Pobil et al., 2000), illustrated in 
Figure 3. In this approach, the robot and the obstacles in the environment are described as a 
set of enveloping spheres 2 . Initially, a small set of large enveloping spheres is used for each 
object. If no intersecting spheres are found, the distance between the two closest sphere 
centers is returned as the distance between the robot and the nearest obstacle or human. If 
two intersecting spheres are found, the robot and the obstacles are decomposed into a set of 
smaller enveloping spheres. The process is repeated until a non-intersecting set of spheres is 
found, or until the maximum level of decomposition is reached, in which case the algorithm 
reports that a collision has been detected. The level of decomposition required to find a 
collision free set of spheres is also used to determine the size of the region within which 
local trajectory planning may be executed, as in (Brock and Khatib, 2002). 
When defining the enveloping spheres for the user, the current robot task also becomes important. 
If the goal of the interaction is for the robot to approach and/ or contact the user, then it is not 
appropriate to represent the user simply as an obstacle, as in (Traver, del Pobil et al., 2000). Instead, 
in this work, during pre-planning, each segment of the path is classified as interactive or non- 
interactive. If the segment is classified as non-interactive, the entire region of space occupied by the 
user is treated as an obstacle. If the segment is classified as interactive, a smaller set of spheres is 
used, such that the target area of the user (for example, the hand) is excluded from the obstacle area. 
By only excluding the contact area, this approach ensures that the robot can reach the intended 
target, while motion is still restricted to non-target areas of the body. Using this representation also 
ensures that the robot will slow down as it approaches the target, due to the effect of nearby 
spheres. Figure 3 (b) shows the representation during an interactive task segment. 





(a) 



Fig. 3. (a) Human, robot representation in a 
representation in an interactive task. 



(b) 
non-interactive 



task, (b) Human, robot 



2 Other representations could also be used (for example, blobs) (Wren, Azarbayejani et al., 1997), however, this 
approach provides fast computation and the necessary accuracy level required 
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2.2 The Overall Cost Function 

The planning-cost function is generated by combining the goal seeking, obstacle avoidance, 
and danger criteria. The planned path is generated by searching for a set of configurations 
that minimize the cost function: 



j =W G .f G (D G ) + W .f (D ) + W D .K-DC 



(11) 



Here, Wg is the weighting of the goal seeking criterion, Wo is the weighting of the obstacle 
avoidance criterion, Wd is the weighting of the danger criterion, and K is a scaling factor. 
The selection of the weight levels is discussed in the following section. 

3. Implementation 

Using the above cost function, it is likely that the danger criterion will conflict with the goal 
seeking criteria during the search, leading to local minima and long search times. To avoid this 
problem, a two-stage search is proposed. In the first stage, maximum priority is placed on 
minirnizing the danger criterion. A threshold is established for determining when an acceptable 
maximum level of danger is achieved. Once a path is found that places the robot below this 
threshold, the second stage of the search is initiated. In this stage, maximum priority is placed on 
the goal-seeking criterion. In the resulting overall path, the robot will spend most of its time in low 
danger regions. One can note that, this approach will not result in the shortest distance path. The 
tradeoff between increased safety and reduced distance can be controlled by modifying the 
threshold where switching from the first stage to the second stage occurs. The two stages are 
implemented by modifying the weighting factors. In the first (danger minimization) stage, the 
danger weighting, Wd is greater than the goal seeking weighting, Wg, while in the second stage, 
Wg is greater than Wd. As long as the relative weights are set in this manner, the algorithm does 
not require tuning of the weight levels when using the product based danger criterion. For the 
sum based danger criterion, if the robot is approaching the person, Wd must be small (0.1 or less) 
in the second stage to avoid interference with the goal attraction criterion. 

Even when the proposed two stage planning approach is used to minimize the conflict between 
the danger and goal seeking criteria, it is still possible for the goal seeking and the obstacle 
avoidance to conflict in a cluttered environment, or when joint limits are encountered during the 
search. The search time is also extended if the robot needs to reverse configurations during the 
path (for example, from an elbow down starting configuration to an elbow up final configuration). 
In these cases, a circuitous path is often generated, requiring some post-process smoothing 
(Latombe, 1991). In particular, if there are several obstacles positioned close to the robot, it may not 
be possible to complete the stage 1 search within the given threshold. In this case, the user should 
be notified that a safe path cannot be found in the current environment. 

The problem of long search times can also be addressed by taking advantage of particular 
robot geometry, and searching only through joints that affect the end-effector position. For 
example, although the PUMA560 is a 6 DoF robot, only the first 3 joints contribute to the 
gross end-effector movement. After the position path is generated, the remaining 3 joints 
can be used to maintain a desired end-effector orientation, as required by the pay load. 

4. Search Strategy Improvements (Backwards Search) 

The global planning strategy presented above is generally valid for non-redundant as well 
as redundant robots, as well as robots with either prismatic or articulated joints, or mobile 
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robots. This is because the search is conducted forwards from an initial configuration, the 
search steps are generated from that initial configuration, and therefore only forward 
kinematics are required to calculate the workspace potential field functions. 
If an inverse kinematics routine is available for the robot, the algorithm search time can be 
improved by adding a backwards search stage. This addition is useful in those cases when the 
robot goal is in a crowded area, for example when the robot's goal is the user's hand. In this case, 
to get to the goal, the robot must go into an area of higher potential field, since the goal is 
surrounded by obstacles generating a repulsive field. Therefore, the algorithm must perform 
// well-filling ,, to find the path, increasing the search time, and, potentially, resulting in a 
convoluted final path. On the other hand, if the search can be performed backwards, gradient 
descent can be used to find the lowest potential path to the goal, reducing the search time. 
In general, it is always more efficient to search from the cluttered end of the path (Kondo, 
1991; Hwang and Ahuja, 1992). The inverse kinematics routine is used to generate the goal 
configuration, given the goal workspace position and the desired end-effector orientation at 
the goal. The search is then initiated backwards from the goal configuration towards the 
start configuration. Once obstacle influence is minimal, the backwards search stops, and the 
forward search (as described in Section 0) is initiated, with the last configuration of the 
backwards generated path as the goal. This location at which the backwards search stops, 
and the new goal location for the forwards search is named the intermediate location (IL). If 
there are multiple solutions to the inverse kinematics problem, the danger criterion at each 
solution is evaluated, and the solution with the lowest danger criterion is selected. 
For continuity, the algorithm must also ensure that the forwards and backwards-generated 
paths meet at the same point in configuration space. Since the goal and obstacle potential 
fields are defined in the workspace, it is possible for an articulated robot to reach the 
starting point of the backwards path in an incorrect posture (e.g., elbow up vs. elbow down). 
In this case, the two paths cannot be joined by simply generating a spline between the two 
postures. This could cause the robot to move into obstacles or move through a more 
dangerous configuration. Instead, during the initial stage of the forward search, an 
additional "posture" potential function (Brock and Khatib, 2002) is added, that favors the 
starting posture of the backwards path. The posture function is defined as: 
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Here, q is the search configuration joint angle, and quad m { n and quad max are the quadrant 
boundaries centered around the each joint angle found in the final configuration of the 
backwards search (i.e., the joint angles associated with the IL). 

The posture function is calculated for each joint; the total posture function is the sum 
over all the joints. The posture potential is only active while the robot is in the 
incorrect posture. When the robot reaches the correct posture, the posture potential 
becomes inactive. To ensure that the correct posture is reached prior to merging with 
the backwards planned path, an additional condition is added to the switchover from 
the first to the second stage of the planning. Namely, in addition to the low danger 
index requirement, the posture potential must also be zero. A flow chart for the 
complete algorithm, including the backwards search, is shown in Fig. 4. 
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Fig. 4. Combined backwards-forwards search algorithm flowchart. 



5. Simulations 

A simulation environment was developed to test the planning algorithms with various 
robot architectures. The robots are modeled using the Robotics Toolbox (Corke, 1996). 
Fig 5 shows the planned motion of a 3 link planar robot using the basic algorithm (i.e., 
without the backwards search), with the sum-based danger criterion. The robot's goal 
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is to pick up the object being held by the user. The same task is shown as planned by 
the product-based danger criterion in Fig. 6. In both cases, to better illustrate the effect 
of the danger criterion, only the goal and danger criterion cost functions are included. 
The cost function weights used for both plans are given in Table 1. Fig. 7 shows a 
comparison between the user-robot center of mass distance and the robot inertia for 
the sum-based and the product-based danger criteria. 
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Fig 5. Planned path with sum-based danger criterion. 
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Fig. 6. Planned path with product-based danger criterion. 

Fig 5 and Fig. 6 illustrate the differences between the two danger criterion 
formulations. The sum-based danger criterion implies that the factors affecting the 
danger can be considered independently of one another when assessing the level of 
danger. One advantage of the sum-based criterion is that the formulation is similar to 
other quadratic cost functions normally used in the potential field approach, and is 
distance based. Therefore, the sum-based criterion does not need to be scaled when 
combined with the other criteria (i.e., K = 1). The center of mass distance factor is a 
repulsive potential field, and can, therefore, become infinite in magnitude when the 
center of mass distance between the robot and the user (Dcm) is close to the minimum 
safe distance (D m ; n ). Thus, when the robot and the user are close together, the distance 
factor will dominate over the inertia factor. This effect is illustrated in the last frame of 
the Fig 5 sequence. 
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Table 1. Planar robot simulations weights. 
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Fig. 7. Comparison between the sum-based and product-based danger criteria. 

As a result, near the point of interaction, the sum based criterion results in a higher inertia, 
as can be seen in Fig. 7. In general, the disadvantage of such a sum based formulation is that 
one of the factors always tends to dominate the others. Furthermore, for the sum-based 
criterion it is difficult to define the threshold at which one should switch from the danger 
minimization stage to the goal seeking stage, since the danger criterion is a combination of 
the robot link distances from the robot base and the distance from the robot to the person. 
The product-based danger criterion implies that the factors affecting the danger criterion need to 
be considered collectively when assessing the level of danger. For example, if the distance between 
the robot and the person is large, the other contributing factors will not be minimized either. In 
Fig. 6, since the distance between the robot and the person is small, both the distance factor and 
the inertia factor are minimized. In addition, when both factors have significant magnitude, the 
danger criterion gradient will be steepest, ensuring that the danger criterion is prioritized over the 
other criteria. Because the two factors scale each other, both are minimized to achieve the required 
safety level. Another advantage of the product-based criterion is that the criterion represents a 
clear indication of the level of danger, ranging from to 1 (values greater than 1 are possible when 
the distance between the robot and the user (Dcm) is smaller than the minimum safe distance 
(DcMmin)- Therefore, it is easier to specify the switch threshold as the desired level of danger. 
However, for the product-based criterion, a scaling factor (K) must be chosen so that the danger 
criterion is on the same scale as the goal and obstacle criteria. 

In the majority of cases, the product-based danger criterion is more suitable. The product-based 
criterion is more suitable for redundant robots, where both the inertia and Centre of Mass (CoM) 
distance factors can be minimized, regardless of the CoM distance. When the robot is close to the 
person, the product-based danger criterion will decrease inertia and increase CoM distance. On 
the other hand, close to the person, the sum-based danger criterion becomes dominated by the 
distance factor, so inertia is not reduced as significantly. The sum-based danger criterion may be 
more suitable with large, under-articulated robots. In this case, the difference between the 
maximum and minirnurn robot inertia may not be very significant, whereas the strong CoM 
distance action will ensure that the robot does not get too close to the user. 

Figure. 8. shows a planned motion sequence with a PUMA 560 robot, using the basic 
algorithm with the goal, obstacle and danger criteria. The product based danger criterion is 
used. Table 2 gives the weights used in the search. For comparison, a path was generated 
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using the best-first planner without any danger criterion. As illustrated in Fig. , the danger 
criterion pushes the CoM of the robot away from the person along the majority of the path, 
as well as significantly reducing the robot's inertia. 
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Fig. 8. Planned sequence for a PUMA560 robot (product danger criterion). 



Frame #80 





W G 


Wo 


W D 


Stage 1 


0.1 


0.2 


0.7 


Stage 2 


0.7 


0.2 


0.1 



Table 2. PUMA560 simulations weights. 

Figure 10. shows a planned sequence using the modified algorithm, with the 
backwards search added. In this case, the initial robot pose is the reverse of the 
required final pose generated by the backwards plan. The same weights were used as 
for the basic algorithm, as specified in Table 2. 
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Fig. 9. Effect of the danger criterion search on the danger factors (product danger criterion). 
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Fig. 10. Planned sequence for a PUMA560 robot with backwards search (product danger criterion). 
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Initially, while the danger is low, the posture function dominates the potential field, and the robot 
moves first to move to the correct posture. As the robot comes closer to the person, the danger 
criterion begins to dominate the potential field, and the robot inertia is reduced. Once the danger 
index has been reduced, the robot moves towards the goal. Posture correction is performed during 
low danger sections of the path. As discussed in Section and shown in the flowchart in Fig. 4, the 
backwards search is only performed when the goal location is within the influence distance of 
obstacles. In this case, the basic planner must find the entrance into an obstacle region. Using the 
backwards search, finding a path from the obstacle enclosed goal location to a free region is much 
easier (Kondo, 1991). Once a configuration free from obstacle influence is found through the 
backwards search, the forwards search, incorporating the danger criterion, is initiated to this 
configuration. The posture potential must then be added to the forwards search cost function to 
ensure that the forwards and backwards paths join at the same robot configuration. This allows 
the planner goal and obstacle fields to be defined in the workspace, while still ensuring a 
contiguous path in the joint space. 

6. Experiments 

The planner was implemented and tested in a human-robot interaction trial (Kulic and 
Croft, 2006a). The experiment was designed to generate various robot motions and to 
evaluate both the human subjective response and physiological response to the motions. In 
this section, the subjects' subjective evalutation of the robot motion is described. 

6.1 Experimental Method 

The experiment was performed using the CRS A460 6 degree of freedom (DoF) 
manipulator, shown in Figures 11 - 14. The CRS A460 is a typical laboratory scale robot 
with a payload of 1kg, which is suitable for performing table top assistive activities. A 
group of 36 human subjects were tested; 16 were female and 20 were male. The age of the 
subjects ranged from 19 to 56, with an average age of 29.2. Approximately half of the 
subjects were recruited from the students and staff of the Mechanical Engineering 
Department and the University of British Columbia, and the other half were recruited 
from off campus. The subjects were also asked to rate their familiarity with robots on the 
Likert scale, with 1 indicating no familiarity, and 5 indicating excellent familiarity. Of the 
36 subjects, 17 had little or no familiarity with robots (response of 1 or 2), 11 had 
moderate familiarity (response of 3) and 7 had high familiarity (response of 4 or 5). Each 
subject was tested once over a contiguous time period of approximately 25 minutes. 
Single trials of multiple subjects were selected over multiple trials of a single subject in 
order to capture a general response to the robot motions. 

6.1.1 Trajectory Generation 

Two different tasks were used for the experiment: a pick-and-place motion (PP), similar to the 
trajectory displayed to subjects in Nonaka et al. (Nonaka, Inoue et al., 2004) and a reach and retract 
motion (RR). These tasks were chosen to represent typical motions an articulated robot 
manipulator could be asked to perform during human-robot interaction, for example during 
hand-over tasks. For the pick-and-place motion, the pick location was specified to the right and 
away from the subject, and the place location was directly in front and close to the subject. For the 
reach and retract motion, the reach location was the same as the place location. For both tasks, the 
robot started and ended in the "home" upright position. The main difference between the two 
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tasks is the approach direction of the robot. For the PP task, the robot approaches the subject from 

the side, while during the RR motion, the robot approaches the subject from head on. 

Two planning strategies were used to plan the path of the robot for each task: a conventional 

potential field (PF) method with obstacle avoidance and goal attraction (Khatib, 1986), and 

the safe path method (S) described in Section 2 above. Fig, Fig, Fig. and Fig. show frames of 

video data depicting each motion type. 

Given the path points generated for each task by the two planners, a motion trajectory was 

generated using a minimum time cubic trajectory planner planning in configuration space. 

For each path, trajectories at three different speeds were planned (slow, medium, fast), 

resulting in 12 trajectories. 

The trajectory planner generated a set of cubic path segments between each set of path 

points, resulting in a trapezoidal acceleration profile respecting velocity, acceleration and 

jerk limits. Each segment was described by a cubic polynomial, as shown in Equation 13. 



Q(r) = b +b 1 r + b 2 r 2 +b 3 r 3 



(13) 



Where r is the parameterized time, bi are the cubic coefficients, and Q is the resulting joint 
trajectory. 

r = kt (14) 

where t is time and k is a constant scaling factor, which scales the velocity, acceleration and 
jerk on the trajectory. To generate the slow, medium and fast trajectories, k = 0.1, 0.5 and 1.0 
were used, respectively. For both tasks and both planners, the robot comes to a stop in front 
of the person. The peak speeds along the path for both planners are the same, but because of 
dynamic and kinematic constraints of the robot, the velocity along the path cannot be 
identical, however the average speed along the path is within 10% for the two planners. 
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Fig. 11. Path PP-PF (Pick and Place Task Planned with the Potential Field Planner). 
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Fig. 12. Path PP-S (Pick and Place Task Planned with the Safe Planner). 
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Fig.13. Path RR-PF (Reach and Retract Task Planned with the Potential Field Planner). 
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Fig. 14. Path RR-S (Reach and Retract Task Planned with the Safe Planner). 

6.1.2 Experimental Procedure 

For each experiment, the subject was asked to read a description of the experiment and sign a 
consent form. After signing the consent form, the experimental protocol was explained to the 
subject, and physiological sensors attached. The human subject was seated facing the robot. The 
robot then executed the 12 trajectories described above. The trajectories were presented to each 
subject in randomized order. After each trajectory had executed, the subject was asked to rate their 
own response to the motion in the following affective response categories: anxiety, calm and 
surprise. The Likert scale (from 1 to 5) was used to characterize the response, with 5 representing 
"extremely" or // completely ,/ and 1 representing "not at all". 

6.2 Results 

The subject reported responses were analyzed to determine how the various robot motions 
affected the subject's perceived anxiety, calm and surprise, and to determine if the safe 
planned motions were perceived to be less threatening. Fig. and Fig. show a comparison of 
the average responses between the potential field and the safe planned paths for the subject 
rated anxiety and surprise, respectively. 

As expected, for each trajectory, there is a strong positive correlation between anxiety and 
speed, and surprise and speed, and a negative correlation between calm and speed. 
A comparison of the graphs in Fig. and Fig. indicates that for each motion type (pick and place or 
reach and retract), on average the subjects reported lower levels of anxiety and surprise, and 
higher levels of calm, for the safe planned paths. This observation is confirmed by a three factor 
analysis of variance (ANOVA) performed on the anxiety and surprise responses. The three factors 
are Plan (potential field (PF) vs. safe plan (S)), Task (reach and retract (RR) vs. pick and place (PP)), 
and Speed. Statistically insignificant factors were removed and the data re-analyzed until only 
statistically significant factors remained (Hicks, 1993). The statistically significant factors at p < 0.05 
for anxiety and surprise are shown in Table 3 and Table 4, respectively. For these responses, the 
plan, speed and plan*speed interaction were found to be statistically significant factors. The task 
factor and all the task interaction factors were found to be statistically insignificant. A Levene test 
was performed and confirmed the homogeneity of variances assumption at a significance level of 
0.01%. For the subjective ratings (anxiety, calm and surprise), the results show a statistically 
significant reduction in anxiety and surprise (and increase in calm) when the safe planner is used 
when compared with the generic potential field planner at medium and high speeds. The 
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plan*speed interaction indicates that at low speeds, the plan type does not affect subjective 
response, while at higher speeds, the motion plan significantly affects the perceived anxiety, 
surprise and calm. These relationships are found regardless of the type of task performed. 
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Fig. 15. Subject Reported Average Anxiety Response. 
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Fig. 16. Subject Reported Average Surprise Response. 
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Table 3. ANOVA for Anxiety. 
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Table 4. ANOVA for Surprise. 
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7. Conclusions 

The proposed safe planner reduces the factors affecting danger along the path. Using the two-stage 
planning approach reduces the depth of local minima in the cost function, allowing the planner to 
execute quickly. Minimizing the danger criterion during the planning stage ensures that the robot is 
in a low inertia configuration in the case of an unanticipated collision, as well as reducing the 
chance of a collision by distancing the robot centre of mass from the user. This advance-planning 
approach puts the robot in a better position to deal with real-time safety hazards. 
When an inverse kinematics routine is available for an articulated robot, the performance of 
the planner can be further improved by adding a backwards search. That is, the path is 
generated backwards from the goal when the goal location is in an area crowded by 
obstacles. To ensure that the forwards and backwards generated paths meet, a posture 
potential is added to the total cost function. By including the posture potential directly into 
the cost function, rather than splining the two paths after they are generated, the algorithm 
ensures that posture correction occurs during low-danger sections of the path. 
The proposed method was verified in simulation and experiments. A user study was also carried 
out to determine if the safe planned motion affected the perception of safety by users. Two types 
of robot motions were presented to human subjects during the study: motions planned with a 
conventional potential field planner, and motions planned with the safe planner. Subjects reported 
significantly less anxiety and surprise, and reported feeling more calm when safe planned motions 
were presented, as compared to the conventional potential field planner. 
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1. Introduction 

Robotic solutions have now been implemented in wheelchair navigation (Yanco, 2000), 
hospital delivery (Spiliotopoulos, 2001), robot-assisted navigation for the blind (Kulyukin et 
aL, 2006), care for the elderly (Montemerlo et aL, 2002), and life support partners (Krikke, 
2005). An important problem facing many assistive technology (AT) researchers and 
practitioners is what is the best way for humans to interact with assistive robots. As more 
robotic devices find their way into health care and rehabilitation, it is imperative to find 
ways for humans to effectively interact with those devices. Many researchers have argued 
that natural language dialogue (NLD) is a promising human-robot interaction (HRI) mode 
(Torrance, 1994; Perzanowski et aL, 1998; Sidner et aL, 2006). 

The essence of the NLD claim, stated simply, is as follows: NLD is a promising HRI mode in 
robots capable of collaborating with humans in dynamic and complex environments. The 
claim is often challenged on methodological grounds: if there is a library of robust routines 
that a robot can execute, why bother with NLD at all? Why not create a graphical user 
interface (GUI) to that library that allows human operators to invoke various routines by 
pointing and clicking? Indeed, many current approaches to HRI are GUI-based (Fong & 
Thorpe, 2001). The robot's abilities are expressed in a GUI through graphical components. 
To cause the robot to take an action, an operator sends a message to the robot through a GUI 
event. The robot sends feedback to the GUI and the interaction cycle repeats. 
The NLD proponents respond to this challenge with three arguments. First, it is argued 
that, to humans, language is the most readily available means of communication. If the 
robot can do NLD, the human can interact with the robot naturally, without any of the 
possibly steep learning curves required for many GUI-based approaches. Second, it is 
argued that, from the practical point of view, GUI-based approaches are appropriate in 
environments where the operator has access to a monitor, a keyboard, or some other 
hardware device, e.g., an engineer monitoring a mining robot or an astronaut 
monitoring a space shuttle arm. In some environments, however, access to hardware 
devices is either not available or impractical. For example, an injured person in an 
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urban disaster area is unlikely to communicate with a rescue robot through a GUI. 
Thus, NLD in robots interacting with humans has practical benefits. Finally, an 
argument is made that building robots capable of NLD yields insights into human 
cognition (Horswill, 1995). Unlike the first two arguments made by the NLD camp, this 
argument does not address either the practicality or the naturalness of NLD, the most 
important objective being a cognitively plausible integration of language, perception, 
and action. Although one may question the appropriateness or feasibility of the third 
argument in a specific context, the argument, in and of itself, appears to be sound 
insomuch as the objective is to test a computational realization of a given taxonomy or 
some postulated cognitive machinery (Kulyukin, 2006). This chapter should be read in 
the light of the third argument. 

In this chapter, we will present a robotic office assistant that realizes an HRI approach based 
on gesture-free spoken dialogue in which a human operator engages to make a robotic 
assistant perform various tasks. The term gesture-free is construed as devoid of gestures: the 
operator, due to a disability or a task constraint, cannot make any bodily gestures when 
communicating with the robot. We discuss how our approach achieves four types of 
interaction: command, goal disambiguation, introspection, and instruction-based learning. 
The two contributions of our investigation are a shallow taxonomy of goal ambiguities 
and an implementation of a knowledge representation framework that ties language and 
vision both top-down and bottom-up. Our investigation is guided by the view that, given 
the current state of the art in autonomous robotics, natural language processing, and 
cognitive science, it is not feasible to build robots that can adapt or evolve their cognitive 
machinery to match the level of cognition of humans with whom they collaborate. While 
this remains the ultimate long-term objective of AI robotics, robust solutions are needed 
for the interim, especially for people with disabilities. In our opinion, these interim 
solutions can benefit from knowledge representation and processing frameworks that 
enable the robot and the operator to gradually achieve mutual understanding with respect 
to a specific task. Viewed in this light, human-robot dialogue becomes a method for 
achieving common cognitive machinery in a specific context. Once the operator knows the 
robot's goals and what language inputs can reference each goal, the operator can adjust 
language inputs to suit the robot's recognition patterns. Consequently, the human-robot 
dialogue is based on language pattern recognition instead of language understanding, the 
latter being a much harder problem. 
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Fig. 1. Realization of the 3T skills on the hardware. 
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The scenario discussed in the chapter is the operator using gesture-free spoken dialogue to 
make the robot pick up various pieces of trash, such as soda cans, coffee cups, and crumpled 
pieces of paper, and to carry them to designated areas on the floor. The environment is a 
standard office space where the operator and the robot are not the only actors. The operator 
wears a headset that consists of one headphone and one microphone. 

The chapter is organized as follows. Section 2 presents the system's architecture. Section 3 is 
dedicated to knowledge representation. We discuss the declarative, procedural, and visual types of 
knowledge and how these types interact with each other. Section 4 discusses how command, goal 
disambiguation, introspection, and instruction can be achieved in gesture-free spoken dialogue. 
Section 5 presents some experimental evidence. Section 6 reviews related work. Section 7 states 
several limitations of our approach and discusses future work. Section 8 presents our conclusion. 

2. System Architecture 

The robot is a Pioneer 2DX robot assembled from the robotic toolkit from ActivMedia 
Robotics, Inc. (www.activmedia.com). The robot has three wheels, an x86 200 MHZ onboard 
Windows PC with 32MB of RAM, a gripper with two degrees of motion freedom, and a pan- 
tilt-zoom camera. The camera has a horizontal angle of view of 48.8 degrees and a vertical 
angle of view of 37.6 degrees. Images are captured with the Winnov image capture card 
(www.winnov.com) as 120 by 160 color bitmaps. The other hardware components include a 
CCTV-900 wireless video transmitter, an InfoWave radio modem, and a PSC-124000 
automatic battery charger. Captured images are sent through the video transmitter to an 
offboard Windows PC where the image processing is done. The processing results are sent 
back to the robot through the radio modem. 

The robot has a 3-tier (3T) architecture (Kortenkamp et al., 1996). The architecture consists of 
three tiers: deliberation, execution, and control. The deliberation tier does knowledge 
processing; the execution tier manages behaviors; the control tier interacts with the world 
through continuous control feedback loops called skills. The deliberation tier passes to the 
execution tier task descriptions in the form of sequences of goals to satisfy, and receives 
from the execution tier status messages on how the execution proceeds. The execution tier 
sends commands to the control tier and receives sensory data messages from the control tier. 
The execution tier is implemented with the Reactive Action Package (RAP) system (Firby, 
1989). The system consists of a behavior programming language and an interpreter for 
executing behaviors, called RAPs, written in that language. A RAP becomes a task when its 
index clause is matched with a goal description passed to the RAP system from the 
deliberation tier. Each task is added to the task agenda. To the RAP interpreter, a task 
consists of a RAP, the RAP's variable bindings, the execution progress pointer in the RAP's 
code, and the RAP's execution history. The interpreter continuously loops through the task 
agenda picking up tasks to work on. The interpreter executes the task according to its 
current state. It checks if the success condition of the task's RAP is satisfied. If the success 
condition is satisfied, the task is taken off the agenda. If the success condition is not satisfied, 
the interpreter looks at the step pointed to by the execution progress pointer. If the pointer 
points to a skill, the control tier is asked to execute it. If the pointer points to a RAP, the 
interpreter finds a RAP, checks the success condition and then, if the success condition is not 
satisfied, selects a method to execute. The RAP system runs on-board the robot. If the chosen 
task points to a primitive command, the control tier executes an appropriate skill. The RAPs 
are discussed in further detail in the section on procedural knowledge representation below. 
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The control tier hosts the robot's skill system. The skill system consists of speech recognition 
skills, object recognition skills, obstacle avoidance skills, and skills that control the camera 
and the gripper. The speech recognition and object recognition skills run on an off-board 
computer. The rest of the skills run on-board the robot. The skills running on-board the 
robot interface to Saphira from ActivMedia Robotics, Inc., a software library for controlling 
the Pioneer robot's hardware, e.g., reading sonars, setting rotational and translational 
velocities, tilting and panning the camera, closing and opening the gripper, etc. Skills are 
enabled when the execution tier asks the control tier to execute them. An enabled skill runs 
until it is either explicitly disabled by the execution tier or it runs to completion. The control 
tier is not aware of such notions as success or failure. The success or failure of a skill is 
determined in the execution tier, because both are relative to the RAP that enabled the skill. 
Figure 1 shows the realization of the control tier skills on the system's hardware. 

3. Knowledge Representation 

Our knowledge representation framework is based on the idea commonly expressed in the 
cognitive science and AI literature that language and vision share common memory structures 
(Cassell et al., 1999). In our implementation, this idea is realized by having language and vision 
share one memory structure - a semantic network that resides in the deliberation tier. The semantic 
network provides a unified point of access to vision and action through language and shapes the 
robot's interaction with its environment. It also provides a way to address the symbol grounding 
problem by having the robot interpret the symbols through its experiences in the world. 



3.1 Procedural Knowledge 

The robot executes two types of actions: external and internal. External actions cause the robot to 
sense and manipulate external objects or move around. Internal actions cause the robot to 
manipulate its memory. Objects and actions are referenced by language. Referenced actions 
become goals pursued by the robot. Procedural knowledge is represented through RAPS and 
skills. A RAP is a set of methods for achieving a specific goal under different circumstances. 



(definc-rap 

( l iidex ( ge t- ph y s -ob j ? oJb ] ) ) 
(success (location 76hj gripper) ) 
(npetftoct 
(concexc (obj-Ky ?ebj ?k ?y) ) 
(tasfc-tiet 
£nl (naviga^e-to ?oo3)j 
1 1 Z { y c ob- phy a - ob J ?ofeJ I ) > ) 

[context (not fofcj-xy 7obj 7x ?y) ) } 

(caste- nee 
(LI ( locate- piiy a ~ub j 7 ob j ) ) 
iz2 (navigate -to ?ofcj)J 
(t3 (gtuUy yhyu ubj 7obj)))J) 



Fig. 2. A RAP for getting a physical object. 

As an example, consider the RAP for getting a physical object, i.e., a coffee cup, a soda can or a 
crumpled piece of paper, given in Figure 2. The index clause of this RAP specifies the goal the 
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RAP is supposed to achieve. The symbols that begin with question marks denote variables. The 
success clause specifies the condition that must be true in the RAP memory for the RAP to 
succeed. In this case, the RAP succeeds when the object is in the robot's gripper. The RAP has two 
methods for getting an object. The steps needed to execute a method are referred to as task nets. 
The applicability of each method is specified in its context clause. For example, the RAP's first 
method assumes that the robot knows where the object is, i.e., the x and y coordinates of the object 
relative to the robot. On the other hand, the second method makes no such assumption and 
instructs the robot to locate the object first. For example, (get-phys-obj pepsican) matches the index 
clause of the RAP in Figure 2, unifying the variable lob] with the symbol pepsican. 
Vision is considered a function that extracts restricted sets of symbolic assertions from images. This 
approach has its intellectual origins in purposive vision (Firby et al., 1995): instead of constructing the 
entire symbolic description of an image, the robot extracts only the information necessary for the 
task at hand. For example, detect-obj-skill is enabled when a RAP method needs to detect an object. 
The skill has the following predicate templates associated with it: (detected-obj ?obj), (dist-to lob] ?mt), 
(obj-xy lob] Ix ly), and (sim-score lob] Isc). These templates, when filled with variable bindings, state 
that an object lob] has been detected Imt meters away from the robot, and the bottom left 
coordinates of the image region that had the best matching score of Isc are Ix and ly. The output of 
a skill is a set of symbolic assertions, i.e., predicate templates instantiated with variable bindings and 
asserted in the RAP interpreter's memory. In this way, information obtained from vision percolates 
bottom-up to the deliberation tier and becomes accessible through language. 
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Fig. 3. A part of the robot's DMAP-Net. 



3.2 Declarative Symbolic Knowledge 

The declarative symbolic knowledge is a semantic network of goals, objects, and actions. Figure 3 
shows a small part of the robot's semantic network. Nodes in the network are activated by a 
spreading activation algorithm based on direct memory access parsing (DMAP) (Martin, 1993). 
Since the semantic network uses DMAP, it is referred to as DMAP-Net. The activation algorithm is 
detailed in the section on knowledge access below. Each node in the network is a memory 
organization package (MOP) (Riesbeck & Schank, 1989) and, therefore, starts with the m-prefix. 
The solid lines correspond to the abstraction links; the dotted lines denote the packaging links. For 
example, m-get-phys-ob], which corresponds to the action of getting a physical object, has two 
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packaging links: the first one links to m-get-verb and the second one links to m-phys-obj. A packaging 
link is loosely interpreted as a part-of 'link. The two links assert that m-get-phys-obj has two typed 
slots: m-get-verb and m-phys-obj. Every MOP is either an abstraction or a specification. Since every 
action, when activated, becomes a goal pursued by the robot, every action MOP is a specification 
of m-goal, and, conversely, m-goal is an abstraction of every action MOP. Since the robot acts on a 
small closed class of objects, such as soda cans and crumpled pieces of paper, every goal's MOP 
has two packaging links: action and object. The action link denotes an executable behavior; the 
object link denotes an object to which that behavior is applied. 



3.3 Visual Knowledge 

In addition to symbolic names, objects are represented by model libraries created from images 
taken from different distances and with different camera tilts. Each object has two types of models: 
Hamming distance models (HDMs) and color histogram models (CHMs). The two types of 
models are used by the object recognition algorithms briefly described below. The distance is 
measured in meters from the robot's gripper to the object. The models are created for the following 
distances: 0.5, 1.0, 1.5, and 2 meters. This is because the robot's camera cannot detect anything 
further than 2 meters: the vision system is incapable of matching objects beyond that distance 
because of image resolution. For example, under the current resolution of 120 by 160 pixels, the 
size of a soda can that is more than 2 meters away from the robot is about 3-4 pixels. The vision 
system cannot perform meaningful matches on objects that consist of only a few pixels. 
To create an HDM, an object's image is taken from a given distance and with a given camera 
tilt. The subregion of the image containing the object's pixels is manually cropped. The 
subregion is processed with the gradient edge detection mask (Parker, 1993) and turned into 
a bit array by thresholding intensities to or 1. Figure 4 shows the process of HDM creation. 
The top left image containing a soda can is manually cropped from a larger image. The top 
right is the image after the application of the gradient edge detection mask. The bottom 2D 
bit array is obtained from the top right image by thresholding intensities to and 1. An 
HDM consists of a 2D bit array and two integer row constraints specifying the region of an 
image where a model is applicable. For example, the row constraints of the two meter 
pepsican model are 40 and 45. At run time, if the robot is to find out if there is a pepsican 
two meters away from it, the robot will apply the model only in the subregion of the image 
delineated by rows 40 and 45. Each object has five HDMs: two models for the distance 0.5, 
one with tilt and one with tilt -10, and three models with tilt for the remaining distances. 
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Fig. 4. HDM creation. 
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The CHMs are created for the same distances and tilts as the HDMs. To create a CHM, an 
object's image is taken from a given distance and with a given camera tilt. The region of the 
image containing the object is manually cropped. Three color histograms are created from 
the cropped region. The first histogram records the relative frequencies of red intensities. 
The other two do the same for blue and green. Each histogram has sixteen intensity 
intervals, i.e., the first is from to 15, the second is from 16 to 31, etc. Thus, each CHM 
consists of three color histograms. The same row constraints as for the HDMs hold. 

3.4 Object Model Matching 

Two object recognition techniques are used by the robot: template matching (Young et al., 1994; 
Boykov & Huttenlocher, 1999) and color histogram matching (Swain & Ballard, 1991; Chang & 
Krumm, 1999). Object recognition happens in the control tier, and is part of the robot's skill 
system. HDM and CHM matching run in parallel on the off-board Windows PC connected to the 
robot via wireless Ethernet and a radio modem. A match is found when the regions detected by 
the HDM and the CHM for a given object and for a given distance have a significant overlap. The 
significance is detected by thresholding the matches in the overlap area. 

Our template matching metric is based on a generalization of the Classic Hamming Distance 
(CHD) (Roman, 1992). CHD is an edit distance with the operations of insertion and deletion. 
Given two bit vectors of the same dimension, CHD is the minimum number of bit changes 
required to change one vector into the other. However, since it measures only the exact extent to 
which the corresponding bits in two bit vectors agree, CHD does not recognize the notion of 
approximate similarity. The Generalized Hamming Distance (GHD) is also an edit distance, but the 
set of operations is extended with a shift operation. By extending the operation set with the shift 
operation, we account for the notion of approximate matches when corresponding bits in two bit 
vectors are considered aligned even when they are not in the exact same positions. The 
mathematical properties of GHD are described in Bookstein et al. (2002). A dynamic programming 
algorithm for computing GHD is detailed in Kulyukin (2004). 

Object recognition with HDMs is done as follows. Let Mh (O, D, T, L, U) denote an HDM for 
object O at distance D and with tilt T and two row constraints L and U for the lower and 
upper rows, respectively. Let H and Wbe the model's height and width. Let I be an image. I is 
first processed with the gradient edge detection mask. To recognize O in I, the HDM is 
matched with the region of I whose bottom row is L and whose top row is U + H. The 
matching is done by moving the model from left to right C columns at a time and from top to 
bottom R rows at a time. In our experiments, R=C=1. The similarity coefficient between the 
model and an image region is the sum of the GHDs between the corresponding bit strings of 
the model and the image region normalized by the model's size. The sum is normalized by 
the product of the model's height and width. Similarity results are 4-tuples <x, y, m, s>, where 
x and y are the coordinates of the bottom left corner of the image region that matches the 
model m with the similarity score s. Similarity results are sorted in non-decreasing order by 
similarity scores and the top N matches are taken. In our experiments, N=l. 
Object recognition with CHMs is similar. Let M c (O, D, T, L, U) be the H x W CHM of 
object O at distance D and with tilt T and two row constraints L and 17. Let I be an 
image. To recognize O in I with Mc (O, D, T, L, U), the H xW mask is matched with the 
appropriate region of the image R. The similarity between Mc and R is the weighted 
sum of the similarities between their corresponding color histograms. Let h(Hi, H2) is 
the similarity between two color histograms Hi and H2 computed as the sum of the 
absolute differences of their relative frequencies. Then <= h(Hi, H2) <= Q, where Q is 
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a suitably chosen constant. Illumination invariant color spaces were not used because 
of the constant lighting assumption. 

We do not claim that these metrics are superior to other template or color histogram 
matching metrics. Since our objective is the integration of perception and action, our focus is 
on the integration mechanisms that bring these cognitive processes together. It is our hope 
that, once these mechanisms are established, other object recognition techniques will be 
integrated into the proposed framework. 

3.5 Declarative Knowledge Access 

Knowledge in the DMAP-Net is accessed by processing token sequences obtained from 
speech recognition (Kulyukin & Steele, 2002). For example, when the operator says "Get the 
soda can," the speech recognition engine converts the input into a sequence of four tokens: 
(get the soda can). A token is a symbol that must be directly seen in or activated by the input. 
Token sequences index nodes in the DMAP-Net. For example, m-pepsican is indexed by two 
token sequences: (a pepsican) and (the pepsican). If an input contains either sequence, m- 
pepsican is activated. Token sequences may include direct references to packaging links. For 
example, the only token sequence associated with m-get-phys-obj is (get-verb-slot phys-obj-slot), 
which means that for m-get-phys-obj to be activated, m-get-verb must be activated and then m- 
phys-obj must be activated. 

Token sequences are tracked at run time with expectations (Fitzgerald & Firby, 2000). An 
expectation is a 3-tuple consisting of a MOP, a token sequence, and the next token in the 
token sequence that must be seen in the input. For example, X = <m-pepsican, (a 
pepsican), a> is a valid expectation. The first element of an expectation is referred to as 
its target; the second element is a token sequence associated with the target; the third 
element is the key. When an input token is equal to an expectation's key, the expectation 
is advanced on that token. For example, if the next token in the input is a, X is advanced 
on it and becomes <m-pepsican, (pepsican), pepsican>. If the next input token is pepsican, it 
will cause X to become <m-pepsican, (), null>. When the token sequence becomes empty, 
the target is activated. 

3.6 Connecting Procedural and Declarative Knowledge through Callbacks 

Since the robot acts in the world, the robot's declarative knowledge must be connected to the 
robot's procedural knowledge represented with RAPs. The two types of knowledge are 
connected through callbacks. A callback is an arbitrary procedure that runs when the MOP 
with which it is associated is activated. The MOP activation algorithm is as follows: 

1. Map a speech input into a token sequence. 

2. For each token T in the token sequence, advance all expectations that have T as their key. 

3. For each expectation whose token sequence is empty, activate the target MOP. 

4. For each activated MOP and for each callback associated with the MOP, run the callback. 
This algorithm is lazy and failure-driven in that the robot never initiates an interaction with 
the operator unless asked to achieve a goal. This distinguishes our approach from several 
dialogue-based, mixed-initiative approaches where autonomous agents act proactively (Rich 
et al., 2001; Sidner et al., 2006). We conjecture that when the robot and the operator share 
common goals, the complex problems of generic language understanding, dialogue state 
tracking and intent recognition required in proactive approaches can be avoided. The 
algorithm has the robot pursue a goal when the goal's MOP is activated by the input. Such a 
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MOP is made an instance of m-active-goal. When the goal is achieved, the goal becomes an 
instance of m-achieved-goal, and is delinked from m-active-goal. When the MOP associated 
with the goal is partially referenced by the input, i.e., no token sequences associated with the 
goal's MOP are advanced to the end, it becomes an instance of m-ambiguous-goal. 
In sum, to be operational, the system must have the following pieces of a priori 
knowledge: 1) the semantic network of goals and objects; 2) the library of object 
models; 3) the context-free command and control grammars for speech recognition; 4) 
the library of RAP behaviors; and 5) the library of robotic skills. The DMAP-Net is 
constructed manually and manipulated and modified by the robot at run time. The 
library of object models is constructed semi-automatically. The designer must specify 
the region of an image containing an object. From then on everything is done 
automatically by the system. The context-free command and control grammars are 
constructed manually. However, construction of such grammars from semantic 
networks may be automated (Kulyukin & Steele, 2002). The initial RAP behaviors and 
skills are also programmed manually. 

This knowledge representation ties language and vision both top-down and bottom-up. 
Language and vision are tied top-down, because the semantic network links each object 
representation with its models used by object recognition skills. Thus, the robot's memory 
directly connects symbolic and visual types of knowledge. The bottom-up connection is 
manifested by the fact that each object recognition skill in the control tier has associated with 
it a set of predicate templates that are instantiated with specific values extracted from the 
image and asserted into the robot's memory. 

4. Human-Robot Interaction 

The robot and the operator can gradually achieve mutual understanding with respect 
to a specific activity only if they share a common set of goals. It is the common 
cognitive machinery that makes human-robot dialogue feasible. Since the operator 
knows the robot's goals, he or she can adjust language inputs to suit the robot's level of 
understanding. Since the robot knows the goals to be referenced by the operator, the 
robot's dialogue is based on language pattern recognition instead of language 
understanding, the latter being a much harder problem. 

4.1 Mapping Phonemes to Symbols 

Speech is recognized with Microsoft Speech API (SAPI) 5.1, which is freely available from 
www.microsoft.com/speech. SAPI is a software layer that applications use to communicate 
with S API-compliant speech recognition (SR) and text-to-speech (TTS) engines. SAPI 
includes an Application Programming Interface (API) and a Device Driver Interface (DDI). 
SAPI 5.1 includes the Microsoft English SR Engine Version 5, an off-the-shelf state-of-the-art 
Hidden Markov Model speech recognition engine. The engine includes 60,000 English 
words, which we found adequate for our purposes. Users can train the engine to recognize 
additional vocabularies for specific domains. SAPI couples the Hidden Markov Model 
speech recognition with a system for constraining speech inputs with context-free command 
and control grammars. The grammars constrain speech recognition sufficiently to eliminate 
user training and provide speaker-independent speech recognition. Grammars are defined 
with XML Data Type Definitions (DTDs). 
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Fig. 5. Language recognition process. 

The system receives a stream of phonemes, from the operator. SAPI converts the stream of 
phonemes into a sequence of tokens. For example, if the operator says "turn left twenty/' 
SAPI converts it into the token sequence (turn left twenty). The symbolic sequence is sent to 
the semantic network. The spreading activation algorithm uses the symbolic sequence to 
activate MOPs. In this case, the MOP m-turn-command is referenced. When this MOP is 
referenced, the callback associated with it runs and installs the following goal (achieve (turn- 
left 20)) on the RAP system's agenda. The RAP system finds an appropriate RAP and 
executes it on the robot. The robot issues speech feedback to the user through a text-to- 
speech engine (TTS). Figure 5 shows the language recognition process. 

4.2 Four Types of Interaction 

The algorithm outlined in Section 3.6 allows for four types of interaction that the operator 
and the robot engage in through spoken dialogue: command, goal disambiguation, 
introspection, and instruction-based learning. The selection of the interaction types is not 
accidental. If an autonomous robot is to effectively communicate with a human operator in 
natural language, the robot must be able to understand and execute commands, 
disambiguate goals, answer questions about what it knows, and, when appropriate, modify 
its behavior on the basis of received language instructions. In our opinion, these types of 
interaction are fundamental to the design and implementation of autonomous systems 
capable of interacting with humans through spoken dialogue. We now proceed to cover 
each interaction type. 



4.2.1 Command 

Command is the type of interaction that occurs when the operator's input references a goal in the 
robot's memory unambiguously. For example, when the operator asks the robot to pick up a 
pepsican in front of the robot, the robot can engage in the pickup behavior right away. 
Command is a basic minimum requirement for human-robot dialogue-based interaction in that 
the robot must at least be able to execute direct commands from the operator. 
Let us go through an example. Consider the left image in Figure 6. Suppose the operator's 
command to the robot at this point is "Get the pepsican." One of the token sequences 
associated with m-get-phys-obj is (get-verb-slot phys-obj-slot). To activate this MOP, an input 
must first activate its verb slot and then its physical object slot. The m-get's token sequence is 
(get), thus the token get, the first token extracted from the operator's command, activates m- 
get, which, through spreading activation, activates m-get-verb . The token sequence of the 
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MOP's expectation is advanced one token to the right and becomes (phys-obj-slot) . The 
tokens obtained from "the pepsican" activate mpepsican and, since m-phys-obj is an 
abstraction of m-pepsican, phys-obj-slot is activated. When m-pepsican is activated, the object 
models for pepsicans are brought into the robot's memory through a callback associated 
with m-pepsican unless the models are already in memory due to the previous activities. 
After m-phys-obj is activated, the token sequence of the MOP's expectation becomes empty, 
thus activating the MOP. When the MOP is activated, an instance MOP, call it m-get-phys- 
obj-12, is created under it with values m-get for the verb slot and m-pepsican for the object 
slot. The new MOP is placed under m-active-goal to reflect the fact that it is now one of the 
active goals pursued by the robot. As soon as m-get-phys-obj is activated, its callback installs 
(achieve m-get-phys-obj) on the RAP interpreter's agenda. The RAP for achieving this goal 
checks for instances of the MOP m-get-phys-obj that are also instances of m-active-goal. In this 
case, the RAP finds m-get-phys-obj -12, extracts the m-pepsican value from the MOP's object 
slot and installs (get-phys-obj m-pepsican) on the RAP interpreter's agenda. During execution, 
the RAP enables detect-obj -skill. The skill captures the image shown on the left in Figure 6, 
detects the pepsican (the right image in Figure 6 shows the detection region whitened), and 
puts the following assertions in the RAP memory: (detected-obj pepsican), (dist-to 1.5), (obj-xy 
81 43), and (sim-score .73). These assertions state that the skill detected a pepsican 1.5 meters 
away from the robot, and the bottom left coordinates of the image region that had the best 
matching score of .73 are x=81, y=43. The get-phys-obj RAP invokes the homing skill. The 
skill causes the robot to navigate to the object. When the robot is at the object, the RAP 
enables the pickup skill so that the robot can pick up the can with the gripper. Finally, the 
RAP removes m-get-phys-obj -12 from under m-active-goal and places it under m-achieved-goal. 







Fig. 6. Command Execution. 

4.2.2 Goal Disambiguation 

The system is designed to handle three types of goal ambiguity: sensory, mnemonic, and 
linguistic. Sensory ambiguity occurs when a goal referenced by the input is ambiguous with 
respect to sensory data. Mnemonic ambiguity takes place when the input references a goal 
ambiguous with respect to the robot's memory organization. Linguistic ambiguity arises 
when the linguistic input itself must be disambiguated to reference a goal. 
A goal is ambiguous with respect to sensory data when it refers to an object from a detected 
homogeneous set or when it refers to an abstraction common to all objects in a detected 
heterogeneous set. To understand sensory ambiguity, consider a situation when the robot 
has detected two pepsicans and the operator asks the robot to bring a pepsican. The 
homogeneous set of detected objects consists of two pepsicans. The input refers to a 
pepsican, i.e., an object from the detected homogeneous set. All things being equal, the robot 
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always chooses to act on objects closest to it. Consequently, the robot disambiguates the 
input to pick up the closest pepsican, i.e., the closest object from the detected homogeneous 
set. This disambiguation strategy does not require any human-robot dialogue. 
A goal is mnemonically ambiguous when it references an abstraction with multiple specifications. 
To understand mnemonic ambiguity, consider a situation when the robot receives the input "Get a 
soda can" after it has detected one pepsican and two root beer cans. In this case, the set of detected 
objects is heterogeneous. The operator's input references an abstraction common to all objects in 
the set, because each object is an instance of m-soda-can. Since there is no way for the robot to guess 
the operator's intent with respect to the type of soda can, the robot initiates a dialogue to have the 
operator commit to a specific object type in the detected set. In this case, the robot asks the 
operator what type of soda can it needs to pick up. 

The difference between sensory ambiguity and mnemonic ambiguity is that the former is post- 
visual, i.e., it occurs with respect to objects already detected, i.e., after the sensing action has been 
taken, whereas the latter is pre-visual, i.e., it occurs before a sensing action is taken. To appreciate the 
difference, consider a situation when the operator asks the robot to find a soda can. Assume that the 
robot does not have any sensory data in its memory with respect to soda cans. The input references 
an abstraction, i.e., m-soda-can, that, according to Figure 3, has two specifications: m-pepsican and m- 
rootbeer-can. Thus, the robot must disambiguate the input's reference to one of the specifications, i.e., 
choose between finding a pepsican or a root beer can. This decision is pre-visual. 
Mnemonic ambiguity is handled autonomously or through a dialogue. The decision to 
act autonomously or to engage in a dialogue is based on the number of specifications 
under the referenced abstraction: the larger the number of specifications, the more 
expensive the computation, and the greater the need to use the dialogue to reduce the 
potential costs. Essentially, in the case of mnemonic ambiguity, dialogue is a cost- 
reduction heuristic specified by the designer. If the number of specifications under the 
referenced abstraction is small, the robot can do a visual search with respect to every 
specification and stop as soon as one specification is found. 

In our implementation, the robot engages in a dialogue if the number of specifications 
exceeds two. When asked to find a physical object, the robot determines that the number of 
specifications under m-phys-obj exceeds two and asks the operator to be more specific. On 
the other hand, if asked to find a soda can, the robot determines that the number of 
specifications under m-soda-can is two and starts executing a RAP that searches for two 
specification types in parallel. As soon as one visual search is successful, the other is 
terminated. If both searches are successful, an arbitrary choice is made. 

One type of linguistic ambiguity the robot disambiguates is anaphora. Anaphora is a 
pronoun reference to a previously mentioned object (Jurafsky & Martin, 2000). For example, 
the operator can ask the robot if the robot sees a pepsican. After the robot detects a pepsican 
and answers affirmatively, the operator says "Go get it." The pronoun it in the operator's 
input is a case of anaphora and must be resolved. The anaphora resolution is based on the 
recency of reference: the robot searches its memory for the action MOP that was most 
recently created and resolves the pronoun against the MOP's object. In this case, the robot 
finds the most recent specification under m-detect-object and resolves the pronoun to refer to 
the action's object, i.e., a pepsican. 

Another type of linguistic ambiguity handled by the system is, for lack of a better term, 
called delayed reference. The type is detected when the input does not reference any goal 
MOP but references an object MOP. In this case, the input is used to disambiguate an active 
ambiguous goal. The following heuristic is used: an input disambiguates an active 
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ambiguous goal if it references the goal's object slot. For example, if the input references tri- 
color and an active ambiguous goal is m-get-soda-can, whose object slot does not have the 
color attribute instantiated, the referenced m-color is used to instantiate that attribute. If the 
input does not refer to any goal and fails to disambiguate any existing goals, the system 
informs the operator that the input is not understood. To summarize, the goal 
disambiguation algorithm is as follows: 

• Given the input, determine the ambiguity type. 

• If the ambiguity type is sensory, then 

o Determine if the detected object set is homogeneous or heterogeneous. 

o If the set is homogeneous, choose the closest object in the set. 

o If the set is heterogeneous, disambiguate a reference through dialogue. 

• If the ambiguity type is mnemonic, then 

o Determine the number of specifications under the referenced abstraction, 

o If the number of specifications does not exceed two, take an image and 

launch parallel search, 
o If the number of specifications exceeds two, disambiguate a reference 

through dialogue. 

• If the ambiguity type is linguistic, then 

o If the input has an impersonal pronoun, resolve the anaphora by the 

recency rule, 
o If the input references an object MOP, use the MOP to fill an object slot of 

an active ambiguous MOP. 

• If the ambiguity type cannot be determined, inform the operator that the input is not 
understood. 

4.2.3 Introspection and Instruction-Based Learning 

Introspection allows the operator to ask the robot about it state of memory. 
Introspection is critical for ensuring the commonality of cognitive machinery between 
the operator and the robot. Our view is that, given the current state of the art in 
autonomous robotics and natural language processing, it is not feasible to build robots 
that can adapt or evolve their cognitive machinery to match the level of cognition of 
humans with whom they collaborate. While this remains the ultimate long-term 
objective of cognitive robotics, robust solutions are needed for the interim. 
Introspection may be one such solution, because it is based on the flexibility of human 
cognitive machinery. It is much easier for the operator to adjust his or her cognitive 
machinery to the robot's level of cognition than vice versa. Introspection is also critical 
for exploratory learning by the operator. For, if an application does not provide 
introspection, learning is confined to input-output mappings. 

One important assumption for introspection to be successful is that the operator must know 
the robot's domain and high-level goals. Introspective questions allow the operator to 
examine the robot's memory and arrive at the level of understanding available to the robot. 
For example, the operator can ask the robot about physical objects known to the robot. 
When the robot lists the objects known to it, the operator can ask the robot about what it can 
do with a specific type of object, e.g., a soda can, etc. The important point here is that 
introspection allows the operator to acquire sufficient knowledge about the robot's goals 
and skills through natural language dialogue. Thus, in principle, the operator does not have 
to be a skilled robotic programmer or have a close familiarity with a complex GUI. 
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Fig. 7. Introspection and Learning. 

We distinguish three types of introspective questions: questions about past or current state (What 
are you doing? Did/ Do you see a pepsican?), questions about objects (What objects do you know 
of?), and questions about behaviors (How can you pick up a can?). Questions about states and 
objects are answered through direct examination of the semantic network or the RAP memory. 
For example, when asked what it is doing, the robot generates natural language descriptions of the 
active goals currently in the semantic network. Questions about behaviors are answered through 
the examination of appropriate RAPs. Natural language generation is done on the basis of 
descriptions supplied by the RAP programmer. We extended the RAP system specification to 
require that every primitive RAP be given a short description of what it does. Given a RAP, the 
language generation engine recursively looks up descriptions of each primitive RAP and 
generates the RAP's description from them. For example, consider the left RAP in Figure 7. When 
asked the question "How do you dump trash?", the robot generates the following description: 
"Deploy the gripper. Move forward a given distance at a given speed. Back up a given distance at 
a given speed. Store the gripper." 

Closely related to introspection is instruction-based learning (Di Eugenio, 1992). In our system, this 
type of learning occurs when the operator instructs the robot to modify a RAP on the basis of the 
generated description. The modification instructions are confined to a small subset of standard 
plan alternation techniques (Riesbeck & Schank, 1989): the removal of a step, insertion of a step, 
and combination of several steps. For example, given the above description from the robot, the 
operator can issue the following instruction to the robot: "Back up and store the gripper in 
parallel." The instruction causes the robot to permanently modify the last two tasks in the dump- 
trash RAP to run in parallel. The modified rap is given on the right in Figure 7. This is an example 
of a very limited ability that our system gives to the operator when the operator wants to modify 
the robot's behavior through natural language. 

5. Experiments 

Our experiments focused on command and introspection. We selected a set of tasks, defined 
the success and failure criteria for each task type, and then measured the overall percentage 
of successes and failures. There were four types of tasks: determination of an object's color, 
determination of proximity between two objects, detection of an object, and object retrieval. 
Each task required the operator to use speech to obtain information from the robot or to 
have the robot fetch an object. The color task would start with questions like "Do you see a 
red soda can?" or "Is the color of the coffee cup white?" Each color task contained one object 
type. The proximity task would be initiated with questions like "Is a pepsican near a 
crumpled piece of paper?" Each proximity task involved two object types. The proximity 
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was calculated by the robot as a thresholded distance between the detected object types 
mentioned in the input. The object detection task would be initiated with questions like "Do 
you see a soda can?" or "Do you not see a root beer can?" and mentioned one object type 
per question. The object retrieval task would start with commands to fetch objects, e.g., 
"Bring me a pepsican," and involved one object type per task. 

The experiments were done with two operators, each giving the robot 50 commands in each of 
the task categories. Hence, the total number of inputs in each category is 100. In each experiment, 
the operator and the robot interacted in the local mode. The operator would issue a voice input to 
the robot and the robot would respond. Both operators, CS graduate students, were told about 
the robot's domain, i.e., but did not know anything about the robot's knowledge of objects and 
actions, and were instructed to use introspection to obtain that knowledge. Introspection was 
heavily used by each operator only in the beginning. Speech recognition errors were not counted. 
When the speech recognition system failed to recognize the operator's input, the operator was 
instructed to repeat it more slowly. Both operators were told that they could say the phrase 
"What can I say now?" at any point in the dialogue. The results of our experiments are 
summarized in Table 1. The columns give the task types. 
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Table 1. Test Results. 

The color, proximity, and detection tasks were evaluated on a binary basis: a task was a 
success when the robot gave a correct yes/ no response to the question that initiated the task. 
The responses to retrieval tasks were considered successes only when the robot would 
actually fetch the object and bring it to the operator. 
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Fig. 8. Initial dialogue. 

The first row of the table gives the total number of successes in each category. The second 
row contains the number of accurate responses, given that the vision routines extracted the 
correct sets of predicates from the image. In effect, the second row numbers give an estimate 
of the robot's understanding, given the ideal vision routines. As can be seen the first row 
numbers, the total percentage of accurate responses, was approximately 80 percent. The 
primary cause of object recognition failures was radio signal interference in our laboratory. 
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One captured image distorted by signal interference is shown in Figure X. The image 
contains a white coffee cup in the center. The white rectangle in the bottom right corner 
shows the area of the image where the vision routines detected an object. In other words, 
when the radio signal broadcasting the captured image was distorted, the robot recognized 
non-existing objects, which caused it to navigate to places that did not contain anything or 
respond affirmatively to questions about the presence or absence of objects. 
The percentage of successes, given the accuracy of the vision routines, was about 98 percent. 
Several object retrieval tasks resulted in failure due to hardware malfunction, even when the 
vision routines extracted accurate predicates. For example, one of the skills used by the fetch- 
object RAP causes the robot to move a certain distance forward and stop. The skill finishes 
when the robot's speed is zero. Several times the wheel velocity readers kept returning non- 
zero readings when the robot was not moving. The robot would then block, waiting for the 
moving skill to finish. Each time the block had to be terminated with a programmatic 
intervention. The object recognition skills were adequate to the extent that they supported 
the types of behaviors in which we were interested. 

Several informal observations were made during the experiments. It was observed that, as 
the operators 7 knowledge of the robot's memory increased, the dialogues became shorter. 
This appears to support the conjecture that introspection is essential in the beginning, 
because it allows human operators to quickly adjust their cognitive machinery to match the 
cognitive machinery of the robotic collaborator. Figure 8 shows a transcript of an initial 
dialogue between the robot and an operator. 




Fig. 9. Signal interference. 
6. Related Work 

Our work builds on and complements the work of many researchers in the area of human-robot 
interaction. Chapman's work on Sonja (Chapman, 1991) investigated instruction use in reactive 
agents. Torrance (1994) investigated natural language communication with a mobile robot 
equipped with odometry sensors. Horswill (1995) integrated an instruction processing system 
with a real-time vision system. Iwano et al. (1996) analyzed the relationship between semantics of 
utterances and head movements in Japanese natural language dialogues and argued that visual 
information such as head movement is useful for managing dialogues and reducing vague 
semantic references. Matsui et al. (1999) integrated a natural spoken dialogue system for Japanese 
with their Jijo-2 mobile robot for office services. Fitzgerald and Firby (2000) proposed a reactive 
task architecture for a dialogue system interfacing to a space shuttle camera. 
Many researchers have investigated human-robot interaction via gestures, brainwaves, and 
teleoperation. Kortenkamp et al. (1996) developed a vision-based gesture recognition system 
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that resides on-board a mobile robot. The system recognized several types of gestures that 
consisted of head and arm motions. Perzanowski et al. (1998) proposed a system for 
integrating natural language and natural gesture for command and control of a semi- 
autonomous mobile robot. Their system was able to interpret such commands as "Go over 
there" and "Come over here." Rybski and Voyles (1999) developed a system that allows a 
mobile robot to learn simple object pickup tasks through human gesture recognition. 
Waldherr et al. (2000) built a robotic system that uses a camera to track a person and 
recognize gestures involving arm motion. Amai et al. (2001) investigated a brainwave- 
controlled mobile robot system. Their system allowed a human operator to use brain waves 
to specify several simple movement commands for a small mobile robot. Fong et al. (2001) 
developed a system in which humans and robots collaborate on tasks through teleoperation- 
based collaborative control. Kruijff et al. (2006) investigated clarification dialogues to resolve 
ambiguities in human-augmented mapping. They proposed an approach that uses dialogue 
to bridge the difference in perception between a metric map constructed by the robot and a 
topological perspective adopted by a human operator. Brooks and Breazeal (2006) 
investigated a mechanism that allows the robot to determine the object referent from deictic 
reference including pointing gestures and speech. 

Spoken natural language dialogue has been integrated into several commercial robot 
applications. For example, the newer Japanese humanoids, such as Sony's SDR and Fujitsu's 
HOAP, have some forms of dialogue-based interaction with human users. According to 
Sony, the SDR-4X biped robot is able to recognize continuous speech from many 
vocabularies by using data processing synchronization with externally connected PCs via 
wireless LANs (www.sony.com). Fujitsu's HOAP-2 humanoid robot, a successor to HOAP- 
1, can be controlled from an external PC, and is claimed to be able to interact with any 
speech-based software that the PC can run. 

Our approach is complementary to approaches based on gestures, brain waves, and GUIs, 
because it is based on gesture-free spoken language, which is phenomenologically different 
from brain waves, gestures, and GUIs. From the perspective of human cognition, language 
remains a means of choice for introspection and instruction-based learning. Although one 
could theorize how these two interaction types can be achieved with brain waves, gestures, 
and GUIs, the practical realization of these types of interaction is more natural in language- 
based systems, if these systems are developed for users with visual and motor disabilities. 
Dialogue-management systems similar to the one proposed by Fong et al. (2001) depend in 
critical ways on GUIs and hardware devices, e.g., Personal Digital Assistants, hosting those 
GUIs. Since these approaches do not use natural language at all, the types of human-robot 
interaction are typically confined to yes-no questions and numerical template fill-ins. 
Our approach rests, in part, on the claim that language and vision, at least in its late stages, share 
memory structures. In effect, it is conjectured that late vision is part of cognition and, as a 
consequence, closely tied with language. Chapman (1991) and Fitzgerald and Firby (2000) make 
similar claims about shared memory between language and vision but do not go beyond 
simulation in implementing them. Horswill's robot Ludwig (Horswill, 1995) also grounds 
language interpretation in vision. However, Ludwig has no notion of memory and, as a system, 
does not seem to offer much insight into memory structures used by language and vision. 
Our approach proposes computational machinery for introspection, i.e., a way for the robot 
to inspect its internal state and, if necessary, describe it to the operator. For example, the 
robot can describe what it can do with respect to different types of objects and answer 
questions not only about what it sees now, but also about what it saw in the past. Of the 
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above approaches, the systems in Torrance (1994), Perzanowski et al. (1998), and Fitzgerald 
and Firby (2000) could, in our opinion, be extended to handle introspection, because these 
approaches advocate elaborate memory organizations. It is difficult to see how the reactive 
approaches advocated by Chapman (1991) and Horswill (1995) can handle introspection, 
since they lack internal states. GUI-based dialogue-management approaches (Fong et al., 
2001) can, in principle, also handle introspection. In this case, the effectiveness of 
introspection is a function of GUI usability. 

7. Limitations and Future Work 

Our investigation has several limitations, each of which, in and of itself, suggests a future 
research venue. First, the experimental results should be taken with care due to the small 
number of operators. Furthermore, since both operators are CS graduate students, their 
performance is unlikely to generalize over people with disabilities. 

Second, our system cannot handle deictic references expressed with pronouns like "here/ 7 
"there," "that," etc. These references are meaningless unless tied to concrete situations in 
which they are used. As Perzanowski et al. (1998) point out, phrases like "Go over there" are 
meaningless if taken outside of the context in which they are spoken. Typically, resolving 
deictic references is done through gesture recognition (Brooks & Breazeal, 2006). In other 
words, it is assumed that when the operator says "Go over there," the operator also makes a 
pointing gesture that allows the robot to resolve the deictic reference to a direction, a 
location, or an object (Perzanowski et al., 1998). An interesting research question then arises: 
how far can one go in resolving deictic references without gesture recognition? In our 
experiments with a robotic guide for the visually impaired (Kulyukin et al., 2006), we 
noticed that visually impaired individuals almost never use deictic references. Instead, they 
ask the robot what it can see and then instruct the robot to go to a particular object by 
naming that object. This experience may generalize to many situations in which the operator 
cannot be at or near the object to which the robot must navigate. Additionally, there is 
reason to believe that speech and gesture may have a common underlying representation 
(Cassell et al., 1999). Thus, the proposed knowledge representation mechanisms may be 
useful in systems that combine gesture recognition and spoken language. 
Third, our system cannot handle universal quantification and negation. In other words, 
while it is possible to ask the system if it sees a popcan, i.e., do existential quantification, it is 
not possible to ask questions like "Are all cans that you see red?" Nor is it currently possible 
for the operator to instruct the system not to carry out an action through phrases like "Don't 
do it." The only way for the operator to prevent the robot from performing an action is by 
using the command "Stop." 

Fourth, our system can interact with only one user at a time. But, as Fong et al. (2001) show, 
there are many situations in which multiple users may want to interact with a single robot. 
In this case, prioritizing natural language inputs and making sense of seemingly 
contradictory commands become major challenges. 

Finally, our knowledge representation framework is redundant in that the RAP descriptions 
are stored both as part of RAPs and as MOPs in the DMAP-Net. This creates a knowledge 
maintenance problem, because every new RAP must be described in two places. This 
redundancy can be eliminated by making the DMAP-Net the only place in the robot's 
memory that has the declarative descriptions of RAPs. In our future work, we plan to 
generate the MOPs that describe RAPs automatically from the RAP definitions. 
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8. Conclusion 

We presented an approach to human-robot interaction through gesture-free spoken 
dialogue. The two contributions of our investigation are a taxonomy of goal ambiguities 
(sensory, mnemonic, and linguistic) and the implementation of a knowledge 
representation framework that ties language and vision both top-down and bottom-up. 
Sensory ambiguity occurs when a goal referenced by the input is ambiguous with respect 
to sensory data. Mnemonic ambiguity takes place when the input references a goal 
ambiguous with respect to the robot's memory organization. Linguistic ambiguity arises 
when the linguistic input itself must be disambiguated to reference a goal. Language and 
vision are tied top-down, because the semantic network links each object representation 
with its models used by object recognition skills. 

Our investigation is based on the view that, given the current state of the art in 
autonomous robotics and natural language processing, it is not feasible to build robots 
that can adapt or evolve their cognitive machinery to match the level of cognition of 
humans with whom they collaborate. While this remains the ultimate long-term objective 
of cognitive robotics, robust solutions are needed for the interim. In our opinion, these 
interim solutions are knowledge representation and processing frameworks that enable 
the robot and the operator to gradually achieve mutual understanding with respect to a 
specific activity only if they share a common set of goals. It is the common cognitive 
machinery that makes human-robot dialogue feasible. Since the operator knows the 
robot's goals, he or she can adjust language inputs to suit the robot's level of 
understanding. Since the robot knows the goals to be referenced by the operator, the 
robot's dialogue is based on language pattern recognition instead of language 
understanding, the latter being a much harder problem. 
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1. Introduction 

Today, robots are expected to provide various services directly to humans in environments, 
this situation has led to the idea of teams consisting of humans and robots working 
cooperatively on the same task. Various names for this type of human-robot cooperation 
system have emerged including human-friendly robots, personal robots, assistant robots 
and symbiotic robots. These robots will continue to be employed also in the 21st century to 
cope with the increase in the elderly and handicapped, the decrease in the birth rate and 
working population and will be introduced into non-industrial areas such as homes and 
offices to make a rich and comfortable life. Such robots as home-use robots, assistance 
robots, and service robots, should deal with diverse tasks (Fujie, Tani, and Hirano 1994; 
Kawamura and Iskarous 1994). One of the specific situations in the non-industrial areas is 
that the robots coexist and help humans in their life environments. The robots, therefore, 
must be with the capability of human-robot coexistence. They can be called "human- 
symbiotic robots" (HSRs). There are various problems to be solved to develop the HSRs. 
Working and moving among humans requires special concerns on the safety issues, the 
safety of human in view of an unexpected collision should be assured. A HSR should weight 
not significantly more than a human, but mechanical compliance of the surface and joints is 
also a necessity. 

In the past two decades, many researchers have studied to reduce in advance the 
unexpected collision accidents between an industrial robot and a human operator by 
isolating robots in work cells that automatically shut down if a person enters; visuals 
(signs, flashing lights), and audio devices would indicate conditions by the operation 
and vision/ sound alarms, and so on. Because these accident precautions approaches are 
difficult to assure human safety in human-robot environments in which an interaction 
between the person and the robot is presupposed, we have to consider other ways to 
deal with human-robot collisions and such contacts can take place anywhere on the body 
of the robot. Therefore, the HSR should be constructed on the basis of a new philosophy 
from that of past robots. Lim and Tanie (2000) proposed the HSR must be constructed for 
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everybody use, with simple structure like home electronic products, human-like 
mobility, human-like compliance, and with the human-friendly user interface. In 
addition, the HSR requires a robustness and compliance to perform a human-robot 
cooperative task. 

The novelty human safety mechanisms, therefore, should be designed and introduced into 
the HSR, which are able to cope with the problem that the produced impact/ collision 
forces are caused by human-robot unconscious contacts. In this chapter, we focus on 
collision force suppression and develop a simple, low-cost, and effective physical 
mechanism using complaint hybrid joints for the human-symbiotic mobile manipulators. 
During expected/ unexpected collisions with their environments, the hybrid joints will 
passively deform to reduce the produced collision force. Moreover, we propose the 
collision-tolerant recovery controls to realize the desired task despite the unexpected 
collisions. In this chapter, we also examine the control method through simulations and 
experiments. 

1.1 Related Research 

Making the robot human-like compliance is one good way of enhancing safety. There exist 
two general strategies to realize robot compliance: active compliance, and passive 
compliance. The active compliance is provided with a sensor feedback to achieve either a 
control of interaction forces or a task-specific compliance of the robot. The passive 
compliance is realized by using passive deformable devices attached to the robot body. 

1.1.1 Active Compliance 

The active compliance methods may be divided into force control and impedance control. Given 
a detailed environment description, a widely adopted method is hybrid position/ force control. 
The 'hybrid 7 characterization should be the simultaneous control of either position or force in a 
given directions, not both. The task space is partitioned into two orthogonal subspaces. The 
scheme allows adjustment of position and force dynamics independently (Raibert & Craig 1981; 
Wedel & Saridis 1988; Anderson & Spong 1987; Schutter & Van Brussel 1988). On the other hand, 
impedance control is to enforce an adjustable mechanical impedance relationship between the 
force and the position error. Proper adjustment of the impedance parameters ensures bounded 
contact forces. The primary merit of impedance control is that it establishes adjustable balanced 
behavior of the system between position errors and external force (Salisbury 1980; Whitney 1977; 
Kazerooni & Waibel 1988; Goldenberg 1987). 

1.1.2 Passive Compliance 

The most past passive compliance methods were based on the robot's structural compliance using 
special devices such as springs and dampers. Whitney (1982) described the usefulness of remote 
center compliance devices for peg-in-hole insertion tasks. Goswami and Peshkin (1993) described 
the use of hydraulic cylinders to provide a passive wrist with programmable accommodation, and 
they showed how accommodation and damping matrices transform between task-space and joint- 
space of passive redundant manipulators. Cutkosky and Wright (1986) extended the number of 
compliance centers available from a passive wrist by adding pressure-controlled and fluid-filled 
bladders. Mills (1990) used hybrid actuators consisting of a DC servo motor paired with a 
pneumatic bladder actuator to vary manipulator stiffness. Lindsay, Sinha, and Paul (1993) used 
rubber elements in the robot wrist to reduce the effect of impacts. Laurin-Kovitz, Colgate, and 
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Carries (1991) described the programmable passive impedance control using antagonistic 
nonlinear springs and binary dampers and showed that the impedance of a robot might be 
controlled by incorporating programmable mechanical elements into the robot's driving system. 
Other researchers have studied collision force attenuation with human safety in mind. Suita et al. 
(1995) and Yamada et al (1997) addressed the human-oriented design approach to developing a 
viscoelastic covering for the passive compliance and set up safety condition that an arm is called 
"safe" if the impact force is in an acceptable pain tolerance limit. Morita and Sugano (1995, 1996) 
and Morita, Shibuya, and Sugano (1998) developed the mechanical compliance adjuster of which 
the compliant springs are mounted in the Wendy robot arm's joints. In their studies, the 
combination of the safety material and the compliance control was described for the effective 
attenuation of collision force produced by an unexpected collision. Lim and Tanie (1999) proposed 
a robot with a passive trunk for human-robot coexistence. The passive trunk is composed of linear 
springs and dampers between a mobile part and an arm. Other different compliant passive 
mechanism is that such as Yoon, Kang, Kim, etc. (2003) developed the passive compliant joint 
composed of a magneto-rheological (MR) damper and a rotary spring, where the rotary spring 
gives the arm compliant property and the damper has been introduced to work as a rotary viscous 
damper by controlling the electric current according to the angular velocity of spring 
displacement. And Li (2004) proposed a novelty complaint passive mechanism, which is different 
from the traditional spring-damper system, the hybrid joint for holonomic mobile manipulators. 

1.2 Overview 

In this chapter, we develop a simple, low-cost, and effective physical mechanism for 
nonholonomic mobile manipulators, which consists of hybrid joint scheme and soft 
material-covering links as human safety structure against collisions. Then we propose 
switching control of hybrid joint, which is capable of compliantly adapting to human's 
motion or force by switching the hybrid joint to the active mode or the passive mode as 
needed depending on the requirement of a given task. Several recovering position controls 
of the end-effector after the collision are also presented. 

This chapter is organized as follows. In Section 2, we present human safety mechanical structure 
for human-symbiotic robot. In Section 3, we propose the corresponding control methods for the 
safety structure. Section 4 describes simulation and experiment studies on collision tolerance 
control of mobile manipulator. The results verify the efficacies of the proposed physical 
mechanism and the control approach. Finally, the conclusion is presented in Section 5. 

2. Robot Structure for Human Robot Symbiosis 

HSR is assumed to share the living and working place with human beings, and therefore, is 
required to have the functions of human-like compliance, workability, mobility, and so on. 
Especially, even if a person causes a collision with the robot, he/she would not be injured. In this 
section, the novelty safety mechanisms are described to provide more reliable HSR. They are 
compliant hybrid joints and soft-covering links that can passively deform in a collision. 

2.1 A Human-Symbiotic Robot 

As the relative works of active compliance have already been discussed (see Section 1), the active 
control methods are also difficult to secure the high level of reliability in safety because the sensors 
may be saddled with essential problems involving dead angles and disturbances. In addition, 
when the control systems fail to function for some reason, a person involved in a collision may be 
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badly injured. The active compliance control approaches use the force/ torque sensors mounted on 
appropriate locations of the manipulator such as the manipulator wrist and joints to realize 
human safety by the compliance of the manipulator when the end-effector of the manipulator 
collides with a human. The problem in these methods, however, will relate to how the compliance 
parameters should be specified. Generally, the manipulator compliance is determined according 
to the task requirements. In case such requirements are competitive with human safety 
requirements, the difficulty of realizing both human safety and task performance will be 
produced. The active compliance methods also have the problem that the compliance of the 
manipulator is achieved on the basis of control software. 

The passive compliance methods making the robot hardware itself compliant will be more 
reliable, compared with the above active compliance approaches. In general, we know the 
human's body consists of skeleton, soft tissue, and skin. The human's arm has passive compliant 
joints and is connected to his or her waist with several joints through the shoulder. In a collision, 
his or her joints and body should make a passive compliant motion. Lim and Tanie (2000) fully 
examined the human reaction to a collision with his or her environment, they thought, first, the 
produced impact/ collision force is absorbed by the skin with a viscoelastic characteristic; second, 
if the collision force exceeds the tolerable limit of the elastic tissue, his or her shoulder and waist 
joints passively move to cope with the collision; finally, if the magnitude of the collision force 
exceeds the friction force developed between his or her feet and the ground, he or she 
unconsciously steps/ slips on the ground to the direction of the collision force and reduces the 
collision force. They were interested in the viscoelastic compliant motion of a human's waist and 
the slipping and stepping motion of his or her feet, and introduced a human friendly robot. The 
slipping and stepping motion are important to attenuating the collision force, however, there 
would be time-delayed for the steps/ slips on the ground compared with the hybrid joint's direct 
passive motion proposed in the chapter. The time delay would make the impact force very large 
when hardware compliance is too insufficient for the environments to improve the reliability of 
human safety, for example, when the relative velocity of two collision objects is fast. And, we 
expect the collision force as minimized as possible. Directly faster passive response/ deformation 
would greatly decrease the interaction force. Therefore, in the chapter, we introduce the 
mechanisms of hybrid joints for human-robot collisions. 

We develop a human-symbiotic robot (HSR) consisting of a mobile base and an arm using 
hybrid joints. The arm of the HSR is covered with viscoelastic covering and equipped with 
hybrid joints that can be switched to the active mode or the passive mode as needed 
depending on the requirement of the given task. The viscoelastic covering is equipped with 
mechanical elements such as springs and dampers. This HSR can deal with the collision 
between its body including the manipulator and environment. In case that the HSR or a 
human causes an unexpected collision with the other, its viscoelastic cover and hybrid joint 
passively deform according to the collision forces like human's flesh and waist. Therefore, it 
will not be seriously hurt due to the effective suppression of the collision forces caused by 
the elasticity of the body covering and the passive mobility of the hybrid joint. 

2.2 Characteristics of Hybrid Joint 

2.2.1 Development of Hybrid Joint 

To provide a reliable robot which is safe to humans, as mentioned in Section 1. A kind of 
hybrid joint mechanism contributes to the reduction of too large collision forces. The hybrid 
joint consists of an electromagnetic clutch between the motor and the output shaft as shown 
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in Fig. 1. The hybrid joint has two exchangeable modes: passive and active mode. When the 
clutch is released, the joint is free and switched to the passive mode, and the free link is 
directly controlled by the coupling characteristics of the manipulator dynamics. When the 
clutch is on, the link is controlled with the motor. 

When the external force is applied to the z'th link of the arm by a human, or due to the collision 
with an object in the working environment, the adjacent joint will be switched by the clutch to 
follow the collision force and the collision link will deform to suppress the impact force. 
However, the end-effector will be largely deviated from the desired trajectory and task execution 
ability would be deteriorated. In order to deal with this problem, we must introduce the recovery 
control of the arm to compensate for the deviation. Until now, there has been no method of 
hybrid joints to control the relation between the manipulator and the contact environment. 




Fig. 1. A hybrid joint. 

2.2.2 Switching Logic of the Hybrid Joints for Collision Force Suppression 

Consider the hardware compliance systems being worthy of notice in collision force attenuation. 
To achieve the human safety for human-robot (H-R) coexistence, we need to design switching 
logic of the hybrid joint to avoid mental/ physical damage to human. Because non-contact 
sensors, such as vision sensors or proximity sensors, have poor image processing capabilities as 
well as the ambiguity of detectable volume in proximity-sensing techniques, which makes it 
difficult to secure a high level of reliability in unexpected collision. Therefore, we have developed 
a method to the extent where H-R contact at its incipient stage can be detected by the contact 
sensors distributed at the link's surfaces which triggers an earlier response to the robot velocity 
reduction by commanding the emergency stop and the impact force attenuated by switching the 
joint's mode. A robot arm is constrained on a horizontal plane (Fig. 2(a)). Suppose, now, that an 
obstacle with constant approaching velocity causes a collision with the manipulator (Fig. 2(a)) 
and the impulse force F is acting on somewhere of the link i as Fig.2 (a), then, the static relation 
between the external force F and joint torques is: 

r F =J T F (q)F (1) 

where the Jacobian matrix Jf describes the differential relation between the displacement of 
joint space position and the position of the point Af in which the force F is acting, it is 
apparent that this force does not influence directly the behavior of the manipulator beyond 
the link i, therefore, the Jacobian matrix Jf has the form: 

J F = \J A 0|wx(w-oJ V 2) 
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where J a is a m*i matrix associated with the manipulator between the base and the point Af 
in which the force F is acting, hence, (2) can be rewritten in the form: 



T F = [J T A F 0, 



(n-i)xm\ 



(3) 



When the zth link make a collision with the object, to intercept the collision force transferring 
along the zth joint to 1st joint, the ith joint is switched to passive mode. Then the zth joint 
force torque is equal to zero and the output force of the link to the human is zero. The 
impulse force can't transfer to the (z-l)th joint by ]a t , the jth (/ < z) joint torque has little 
collision disturbance and the deform of the link attenuates the collision force. 



hybrid joint 




Fig. 2. (a) External force acting on the arm (left); (b) Collision model (right). 



2.2.3 Contact Model 

Beside for the hybrid joint, a viscoelastic material should be covered on the arm for absorbing the 
contact force in the beginning of collision. Therefore, a contact model consisting of the spring- 
damper and the hybrid joint is studies to produce collision or contact forces during an unexpected 
collision or contact between a robot body or manipulator and its environment (Fig.2 (b)). Assume 
a robot arm is constrained on a horizontal plane, after emergency stop and switching joint, for the 
link using the hybrid joint contacting environment; there will be no torque present in a contact 
between the robot and the object. Only forces, therefore, are considered. Assume the passive joint 
can rotate freely, when the collision happen, the hybrid joint is released to separate both collision 
objects, therefore, no friction arise. 

An object comes into collision/ contact with a soft surface. The relationship of the collision 
force, the surface deformation of the robot and the contacting object is given as 

[M]fe] + [C]b]+m[d = (4) 



where 
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m is the mass of human, / is the 



moment of inertia of the link, 5 is the displacement angle of the link, x is the deformation of 
the soft covering, k is the spring coefficient , c is the damp coefficient, I is the distance of the 
collision point to the joint. The collision force F can be approximately described as: 



F = k(x-lS) + c(x-lS) 



(5) 



From (5), we know the collision force is related with the distance I from collision point to 
adjacent joint. In practice, supposing the arm is redundant enough, if the zth link (z>l) itself 
or I is shorter, in order to attenuate the impulse force, we can set a limit distance lu m , if 
/ > / lim , the corresponding hybrid joint is switched, that is, the ith joint, otherwise, the jth 
joint ( j < i- 1 ) is switched to meet the limit distance, the (j+1 )th joint is blocked. 
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3. Control of Nonholonomic Mobile Underactuated Manipulator 

The advantage of mobile manipulator is to improve the flexibility of system and extend the 
workspace, thus it does not need too many links as the manipulator. However, until now, more 
researches have been done to investigate the dynamics of mobile manipulator with the full- 
actuated joints (Tan & Xi 2002, Yamamoto & Fukuda 2002, Jamisola & Ang 2002) and robot 
underactuated manipulators (De Luca 2000). And less research is made about the topic of the 
mobile underactuated manipulator. The system consists of kinematic constraints (nonholonomic 
mobile base) which geometrically restrict the direction of mobility, and dynamic constraints 
(second order nonholonomic constraint) due to dynamic balance at passive degrees of freedom 
where no force or torque is applied. Therefore, the authors proposed several control methods for 
nonholonomic mobile manipulator when its hybrid joints are underactuated. 

3.1 Dynamics 

Considering n DOF redundant manipulator with the hybrid joints mounted on a 
nonholonomic mobile base and supposing the trajectories of the links are constrained in the 
horizontal plane, then the links' gravity G(q)=0. The vector of generalized coordinates q may 
be separated into two sets q = [q v q r ], where q v eR m describes the vector of generalized 
coordinates appearing in the constraints, and q r eR k are free vector of generalized 
coordinates; n=m+k. Therefore, the kinematic constraints can be simplified: 

A v (q v )q v =0 (6) 

with the constraint matrix A v (q v )e R rXm . Then, the model is: 



M n M n 
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, (0 < i < k) 



where [m]g R nxn is the symmetric and positive definite inertia matrix; [c]e R nxn is the centrifugal 
and Coriolis force vector; F represents the external force on the arm ; q and q a denote the vector 
of generalized coordinates of the passive joint and the active joint, respectively. A switching matrix 
Te R kxk corresponding to the k hybrid joints is introduced here. This matrix T is a diagonal matrix 
and the elements in the matrix are either or 1, if the element T u = 1 , the joint control is set to 

active mode; whereas it is set to passive mode if t u = . z v eR m ~ r represents the actuated torque 
vector of the constrained coordinates; e v g R mx ( m - r "> represents the input transformation matrix, 
X e R r is the Lanrange multiplier. 



3.2 Recovery Control Design 

Supposing the arm has n joints. If unexpected collision happens to the zth link of the arm, the zth 
joint is turned into the passive mode, these joints from the (z+l)th joint to the nth joint are 
blocked. The system has controllable (z-1) joints after collision. If the left active DOF (the number 
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of active joints) of system is more than the DOF of workspace, the manipulator is still redundant. 
Therefore the arm could be reconfigured to recover the end-effector's position. Otherwise, the 
mobile base and the arm are cooperated to compensate the end-effector's displacement. 



3.2.1 Recovery Control Using Arm 

If the manipulator is still redundant after collision, that is, the left active joints in the 
horizontal plane is more than or equal to 2, we may configurate manipulator to compensate 
the passive motion of the zth link. The blocked link rotates the zth joint to attenuate the 
impact force and realize the force following. After the collision, the position of the end- 
effector (x effector, y effector) can be obtained as: 



^effector 

y effector 



l t cos(0, + AO) 
l t sin^ + A6>) 



i 



Ij COS (f)j 

Ij sin (f)j 



(8) 



where (xi, y t ) is the initial position of the zth joint in the global frame, ft is the angle of the zth link 
relative to the global frame before the collision. Ad is displacement angle of the contact force 
exerting on the zth link relative to the global. ^ . is the joint angle in the joint workspace. 

All the blocked links could be regards as one link; the coordinates for the equivalent mass 
center G and orientation of the link are expressed as: 
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where (xj, yj) is the coordinate of mass center of link j and 6g is the operational angle of the 
equivalent mass center G in the global frame, ntj is the mass of link j. The P point is called as 
the center of percussion, which can be obtained as: 



K =( Z 7 > +L m * x ) + y 2 j» /2>-a/4^ 



yl 



where Ij is the inertia moment of link ;. 

To compensate the position displacement, a dynamic feedback linearization control is 

adopted (De Luca A. and Oriolo G., 2000). Define the P point Cartesian position (x ,y ) as: 
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where c 6 = cos0,s0 = sin • 

Le-t [xj y t Y =R(0g)\£ ^iY > % = r l > V = <?\> ^ (0g) is the rotation matrix, g is the linear 
acceleration of the P point along the line OP. ui is the linear acceleration of the base of the 
line OP along the normal to its line. Therefore, we obtain as: 
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Z = x p ce G +y p se G +K0 2 G (14) 

. y p [3] cO G -x p [3] sO G 
y p s0 G +x p c0 G 

7] = x p cO G +y p sO G (16) 

The initial position and the displacement of end effector are known, utilizing the above 
equations, the end effector could be returned to the desired position with the desired 
velocity during the time T. The inverse kinematics of the manipulator is used to obtain the 
position and velocity of the base of the jth joint (j<i): 

*rW«~W,.4) ( 17 ) 

where f~ l is the inverse kinematics of the manipulator from the ith joint to 1st joint, q rj - 
provide the reference trajectory to jth joint of the manipulator to position of the 1st joint at 
the desired position. The q r will provide the reference trajectory to each actuated joint of 
the arm to the desired position after collision disturbance. An actuator command for the 
robot arm T a can be obtained by using a suitable feedback control system according to (7) 
and (17) as follows: 

T a = M 22 (q d r + K av {q d r -q r ) + K ap {q d r -q r )) + C 22 (18) 

where % G R nax na a nd x e R naxna are the positions during an unexpected collision with the 
moving object and the velocity gain matrices, respectively ; q d and q d are derived 
numerically by first and second differences of the desired joint angles qr respectively. 

3.2.2 Recovery Control Using Mobile Base 

If the number of the arm's active joints meets the following equation: 

n <n , -1 (19) 

active workspace x ' 

where n . is the number of the arm's active joints, n , is the DOF of the 

active J workspace 

workspace. De Luca, Oriolo (2000) solved the problem of the controllability of trajectory 
to the desired state from any initial state, however, the robot had at least two active 
joints, i.e., the arm's join number n > 3 (as descriptions in 3.2.1). Therefore, in this 
chapter, we will consider the case of the join number n < 3 ( that is, n = 1 or n = 2 ) and 
the arm has one underactuted joint. The number of the arm's active joints ( the control 
inputs) is less than that of workspace. By the mobile base, however, under the 
nonholonomic constraint, the methods of position control are proposed using the 
cooperation of the mobile base and the arm to realized controllability of trajectory to the 
desired position from an initial state. 

Assume the motion is constrained in 2D environment; then the whole arm is underactuated 
and demonstrated in Fig. 3, where O is the mass centre of the mobile base; the joint O is (x, 
y) in the base frame; OA is d; cp is the direction angle of the mobile base. 
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Fig. 3. Mobile manipulator with one free link. 

The nonholonomic constraint states that the robot can only move in the direction normal to 
the axis of the driving wheels, i.e., the mobile base satisfies the conditions of pure rolling 
and the no slipping. Two motion controllers of the translational controller and the rotational 
controllers extended from the works of Arai and Tanie (1998) are proposed here for 
controlling the free link. But the difference from Arai and Tanie (1998) is that the motion of 
the free link is realized with the nonholonomic mobile base. 

The nonholonomic constraint can be represented with the state vector q v = [x y (p\ and 
the input velocity v = [v c co c Y in the C space as following: 



cos<p 


dsinp 


v c 


smcp 



dcosp 
1 


co c 



(20) 



The new input variables (u t , u n ) for the joint of the free link, that is, the center O in Fig. 3, 
could be expressed as: 



sin(cp - /3) cos(#? - /3) 
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(21) 



where u t is the acceleration component in the x direction of £L, and the component, u nr 
normal to it. The vector (a 0A , a L0A ) > which represents one acceleration sub- vector along the 
line OA and another normal to it, could be transformed from (20) as: 
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And then, 
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(23) 



where V L , a L and V R , a R are the velocities and accelerations of the left wheel and right 

wheel, respectively. 
From (21-23), we can get, 
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c{<p- $)-?-■ s{<p-/3) s{<p-P) + ?-c{<p-/3) 
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The coordinates of P point ( X , V ), that is, the center of percussion P, can be represented 
by the inputs (m, clr) of the mobile base as: 
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(25) 



Then it can be transformed from as 

P = u n /K ;x p =u t cos P ; y p =u t sin P (26) 

Assuming the free link is translation motion along the x-coordinates of £B, and the initial 
and desired configuration of P point at x-axis are xo and x e . We first plan the trajectory of the 
P point, which is composed of the acceleration and deceleration procedure: 



•^ d — ^1 
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(27) 



for (0 < t < 7/2) and (7/2 <t < 7) respectively, and T is the time period of the trajectory 
segment: 



4(x e -x )/T 2 



when 
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where Xd denotes the instaneous reference trajectory, then x v is the actual trajectory: 

u t = [xj + k v e v + k p e p J/cos/? (28) 

where e v = x d - x p and e p = x d - x p . And using the following to eliminate the error of yp 
and/]: 

u n = -2kfit<mfi + kcos 2 ft • v/a t ( / = 1, 2 ) 



where v is the new input. The converted system is: 
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(29) 



(30) 



where z x = a t tan P ,z 2 = z\ 



v = k x y p + k 2 y p + k 3 z x + k 4 z 2 



(31) 
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when the input v is transformed to the input u n by (29), y p and /? are stabilized to zero. 

If the free link is in rotational motion, the input u n is given to turn the orientation of the link 
to follow the desired trajectory: 

u n = k \Pd + k v e v 



-k p e p \ 



(32) 



where /} denotes the reference of /? on the trajectory, e v = J3 re f - ft and e p = P re f - P . 

The P point position error (e x , e y ) in the base frame can be represented using (e t/ e n ) in the 
link frame £L: 

cos P sin P 
sin P - cos P 



(33) 



Integrating e x = -u t cos P , e y = -u t sin P , we get from (33), 
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(34) 



when the link rotates with constant velocity P >0,the input u t is substituted as t = pt , 
Uf = p 2 v , (34) can be expressed as 



(35) 



where e -\e e de Idr de /drf • The system can be controlled by a linear state 
feedback. If P <0, then T = -pt is applied. 

When the angular acceleration is considered, and the j5 is negligible, if P >0, the input u is 
substituted as T - Jfi t ,u - v , (34) becomes 
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(36) 



where e .. = [ 6( 6fj ^ / j r de n /drf • The system can be controlled by a linear state 



feedback v = j£„ e ... If P <0, T , 



f is applied. 



Give the initial position and the goal position, if the free link has an initial angular velocity, 
the free arm could be rest with the rotation controller by the input (at, cir) of mobile base 
from (32), (35), (36) and then rotate the free link until it is in alignment with the desired 
position, and then mobile base pushes the free link to the desired position using translation 
controller by the input (a L , a R ) from (28), (29), (30). 
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3.2.3 Motion Control Using Virtual Link 

If the number of the arm's active joints meets the following equation: 

n active ~ n workspace ~^ \?') 

where n active is the number of the arm's active joints, n workspace is the DOF of the workspace. 
The arm has one underactuated link as shown in Fig.4. 



Free link 



•+ 



Zb 



*-A 




Driving 
wllKl 

Fig. 4. Mobile manipulator with one free link. 

In this case, we utilize the cooperation of both arm and mobile base to move the end-effector 
to the desired position. The locomotion of the mobile base could augments the DOF of the 
system, but the mobile base is nonholonomic, which makes the mobile base only moving in 
the specified direction, rather than arbitrary direction. We have to plan the specified 
trajectory to cope with the nonholonomic constraint. To solve this problem, we propose a 
method applying a virtual link fixed on the mobile base. 

Two cases are discussed here. If the desired position is in the workspace of the manipulator, that 
is, the position can be reached by the rotation of the base and the active link. We treat line CA as a 
virtual link in Fig. 4. The rotation of mobile base around A is equivalent to one revolve joint. 
To track the desired position, we make full use of the virtual link CA and the active link CO. 
At first, we could get the P point position (x ,y ) from (10) and O point position (x,y) . The 

control method as same as (11) - (17) are used to complete the position control, and an 
actuator command r m = [r A t c ] t for the robot system can be obtained by a suitable 
feedback control system according to (7) as follows: 

*m = M(qc,A + K av (q d C ,A " 4) + K ap (q d C , A -q)) + C (38) 

where M a e i? 3x3 is the symmetric inertia submatrix of the arm which include the virtual link. 

CeR 2xl is the centrifugal and Coriolis force vector of the arm. For the virtual joint, 

K and K av dire the position gain parameter and the velocity 



.and 



fa /r + Tr MR = T A aim t L = -t R • ~ ap 

q"c,A miLl <Jc.a 



gain parameter, respectively; q d c A and q^ A are derived numerically by first and second 
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differences of the desired joint angles respectively. Using this control, the end-effector 
trajectory tracking can be realized despite the motion of the passive link. 
If the desired position is in the workspace of the mobile manipulator, the mobile base has to 
move, we assume the mobile base is substituted with a virtual link, whose length could be 
equal to that of the free link and which augments the DOF of the system, and the link is 
connected to the joint C of the arm with a virtual joint. The trajectory of virtual joint could be 
regarded as the motion of the mobile base. Therefore, the DOF of the whole system is equal 
to the DOF of the workspace. This status of the system is similar with Section 3.2.1. Then, we 
can adopt the same control algorithm as the above section. If the mobile base can track the 
trajectory of the joint, the control can be realized. But we must first plan the trajectory of the 
mobile base from the initial position to the position of the virtual joint. Therefore, there 
exists a control problem how to drive the mobile under underactuated manipulator. 
Based on the translational and rotational controllers for the free link in Section 3.2.2, the free 
link probably has an initial rotation velocity because of switching, the input (tit, u n ) of active 
joint C from (28), (29) is utilized to eliminate this angular velocity to rest the free link. Then 
we recover to the desired position (Although is changed in eliminating angular velocity 
of free link). Then, we adopt the above controllers (32), (35), (36) and (28), (29) to complete 
rotation or translation motion of free link by the mobile base and the left active links. When 
the mobile base reaches the desired point, we regulate the posture of the mobile base 
to 6 ± /r/2 . The second step adopts the (11)-(17) to complete the motion control. 

4. Simulation and Experiment Studies 




Fig. 5. (a) Simulation of three link planar robot (left); (b) Hybrid joint based mobile 
manipulator (right). 

4.1 Hybrid Joint Response Experiment 

The rapid response of hybrid joint, that is, the time of switching modes, is an important impact 
factor to achieve the safety for both robot and human, therefore, a shorter time of switching hybrid 
joint would greatly improve the safety of system. Therefore, we conduct the tests to measure the 
switching time of the hybrid joint as shown in Fig. 1. The status of hybrid joint is switched from 
the active mode to passive mode. The releasing time is shown in the Fig.6. The average response 
time is about 0.045s, if we assume the impact velocity is 0.5m/ s, the deformation of link's surface 
would be 0.0225m. Therefore, it is better that the thickness of soft covering should be more than 
0.05~0.06m (considering the response time of emergency stop). 
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Fig. 6. The switching time of hybrid joint. 

4.2 Impact Simulation 

Assuming we adopt the static/ dynamic thresholds of the human pain tolerance limit 
parameterized by the contact force Fumu=50N in the reference of Yamada's criterion (Yamada, 
Suita, etc., 1997), and the arm is safe only if the impact velocity is less than about 0.25m/ sec in the 
paper (Yamada, Suita, etc., 1997). We execute the impact simulation with a 3-DOF planar two- 
wheeled mobile manipulator using the hybrid joints in the Fig. 5 (a). Assume a human collides 
with the last link. The parameters of the mobile manipulator is mi=3kg; m2=3kg; ra3=5kg; Zi=1.5m; 
/2=lm; h=lvcv, k=2/3m. The stiffness and damping of the arm covered with the soft materials and 
the human are selected as /c=800N.m- 1 , c=100N.s.m 4 . The human's mass is 50kg, and the 
maximum impulse velocity of the obstacle could reach to 0.5m/ s under the Fi; OT /£=50N.The 
evolutions of the impact forces on the different collision points of last link are shown in Fig. 7. We 
can see that the impact forces on difference collision points are attenuated quickly. Compared with 
the results of Lim, Tanie (1999, 2000) and Yoon, Kang, etc. (2003), under the condition of no 
damage to the human, the impact velocity has been improved and the impact forces have been 
decreased. When the collision happens to other links, the inertia of moment is increased; however, 
in the case of hybrid joints, the impact force is dependent on coefficients k and c, which can be 
different values on the different links to meet the limit of impulse force. 



4.3 Impact Experiments and Discussions 

The impact experiments have been done using the hybrid joints in mobile manipulator as shown 
Fig. 5(b). Two joints in the horizontal plane are hybrid type and are used to make experiments. 
As similar as the simulations, the arm is with the hybrid joint and covered with a soft 
covering, whose spring and damper are selected respectively as /c=800N nr 1 , c=100N -s mr 1 . 
To confirm our objectives for simplicity, the impact experiments are conducted using: 1) the 
rigid arm without the hybrid joint ; 2) The rigid arm with the hybrid joint. 
Experiment 1: The human collisions with the last link with the soft covering but without hybrid 
joint, the human moves in approximately constant velocity 0.5m/ s. The collision point on the link 
is 0.3m from the joint. The parameters of the collision link of the arm is M=5kg (the fifth link), 
L=0.3m (the fifth's length), and the collision point is 0.3m to the joint. The contact forces are 
measured with the force/ torque sensor attached to the contacted link of the arm. 
Experiment 2: The experiment condition is similar with experiment 1, but the fifth joint 
equipped with hybrid joint of the arm causes an unexpected collision with the human in 
approximately constant velocity 0.5m/ s. 
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Fig. 7. The profile of contact force (blue line) when object collisions with third link, (a): collision 
point is 0.3m relative to the passive joint; (b): collision point is 0.5m relative to the passive joint; 
(c):collision point is 0.7m relative to the passive joint. The human veolcity is 0.5m/ s. 
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Fig. 8. (a) Contact force of the experiment with soft covering but without hybrid joint (left); 
(b) Contact force of the experiment with soft covering and hybrid joint (right). 

In the experiment 1, for the rigid arm with the soft covering but without hybrid joint, the 
contact force is extremely large, as shown in Fig. 8(a), which is similar as Fig.15 (a) in Lim, 
Tanie (1999). However, the duration time of the impact force more than 50N is 
approximately 0.4s and it is attenuated slowly, because without the fast complaint 
deformation of link (or the deformation takes place relatively slowly), the inertia of human 
makes both objects contacted. It is apparent that it would make damage to the human. 
For the experiment 2, the link has an initial velocity about 0.2rad/s. When the collision takes place, 
the trigger on the link is contacted, the system commands the emergency stop and then the circuit 
releases the hybrid joint, the response of collision force is shown in Fig. 8(b), which shows that the 
contact force is remarkably attenuated, damped more quickly and lasts shorter time compared 
with the previous works of Lim, Tanie (1999) and Yoon, Kang, etc. (2003). But the collision 
experiment result is a little different from the simulations because they are executed in the ideal 
situation where no external disturbances (including the friction and unmodeled dynamics, etc) 
have been considered. Therefore, the hardware experiments prove that mechanism is effective. 



4.4 Recovery Control Simulations 

To thoroughly verify the proposed methods, the recovery control simulations are presented 
for every possible H-R interaction case. In Simulation I, the mobile manipulator is shown in 
Fig.4 (a), the collision happens to 3rd link, then, the end-effector moves from initial position 
(1.49m, 1.67m) to another position (1.3163m, 1.9708m), and the third joint angle changes 
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from 400 to 200 with the angular velocity (0.47rad/s). We have to recover the task position 
as soon as the collision link separates with human. Our intention is to drive the third joint to 
the position (0.5503m, 1.3280m, 200) so that the end-effector goes back to the initial position. 
Since the system is still redundant, we only adopt the recovery control using the arm from 
(11)-(18). The trajectories of the end-effector and the third joint are shown in Fig.9 (a). The 
position errors of end-effector are shown Fig. 9(b). From these two figures, we can see the 
actual trajectory of the end-effector has converged to the desired position. 
In Simulation II, we assume a human collides with the second link, the second joint angle change 
from 40 deg to 20deg, the third joint angle is 90deg and the third joint becomes blocked. The second 
joint is switched to passive mode. And then, we adopt the recovering control using a virtual link. 
We can treat the third and the second link as one link and the equivalent mass center G relative to 
the second joint is 0.7905m. Therefore, we can calculate K= 1.0541 m. The end-effector's initial 
position is (2.5m, 2.5m) with the angular velocity 0.5rad/s, the goal position is (1.9546m, 1.1671m). 
The angular velocity has been first eliminated and the desired angle of passive link has been 
obtained before the regulation process. The control result is shown in Fig.ll. Then the regulation 
process of the mobile base to the virtual joint position is shown in Fig.12. And the errors of y- 
coordinate of P point and p in the regulation course are shown in Fig. 13. Then, recovery control is 
going on. Because the third joint is a blocked joint, we utilize the position feedback controller as the 
same as the Simulation I and the position trajectories of the end-effector and 2nd joint are shown in 
Fig.10. Within 20s, the end-effector recovers the initial position. During the recovering, the trajectory 
of the mobile base is shown in Fig. 12. At last, the error evolution of the end-effector position is 
shown in Fig.14. From the figures, we know the end-effector converge to the desired position. 




Fig. 9. (a)The trajectory of the end-effector and the third joint position; (b) Displacement of 
end-effector. 
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Fig. 10. The actual position trajectory of the mobile manipulator, the circles represent the 
base, the blue points denote the 2nd joint, and the red points denote the end-effecotr, 3rd 
joint is the blocked and is not shown. 
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Fig. 11. Eliminate the angular velocity casued by the collision and keeping the angle of 
passive joint to the desired angle before the recovery control. In this case,the desired angle 
of passive joint is 200 before the recovery. 
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Fig. 12. The trajectory of the mobile base. 
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Fig. 13. Error of y (red line) and p(blue line)in the regulation course. 




Fig. 14. Displacement of the end-effector. 
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4.5 Recovery Control Experiments 

For the mobile manipulator we make experiments as shown in Fig.5 (b), we use the mobile base to 
realize the recover control. The length of the free link is 0.3m and its mass is 5kg, /c=0.185m.The 
free joint is 87.2cm relative to the center of the mobile base. The initial Angle 0.1351 rad with zero 
velocity, the desired goal angle is rad. On the time t=17s, an impact force is put on the link, and 
then the link is with an angular velocity. The rotation angle and the angular velocity of free link 
under control are shown in the Fig.15, and after 20s, it approaches the desired angle. The trajectory 
of the mobile base is shown in Fig.16 and the final position is (4.155cm, -1.1126cm). Then, the 
mobile base turns around the center, and makes the direction angle align with the free link. The 
translation of free link with the mobile base goes on. The expected motion of the free link is to 
translate the passive link along the y-axis from (0cm, 87.2cm,) to the position (0cm, 127.2cm), but 
the initial position of the passive joint is (9.47cm, 87.2cm, 0°), that is, we need to eliminate the x- 
coordinate error to cm and keep the displacement angle of the free link to rad. The both control 
errors results are shown in Fig.17 and Fig.18. And the corresponding trajectory of the mobile base 
is demonstrated in Fig. 19. Due to the movement error of the mobile base, the position of the 
mobile base only arrives the (-0.219cm, 40.6cm), but the end-effector has arrived the position (- 
0.219cm, 127.8cm, 0°) in the tolerant error. 
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Fig. 15. The rotation of passive link ,the desired angle p is rad. 
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Fig. 16. The trajectory of mobile base for the rotation p. 
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Fig. 17. Angle of the passive link by the mobile base, the desired J3 is rad. 
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Fig. 18. Error of the passive joint,the desired x coordinate is Om. 
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Fig. 19. The trajectory of mobile base for the translation. 
5. Conclusions 

A hybrid joint scheme has been suggested in order to assure the safe interaction in human-robot 
cooperative tasks. During an unexpected collision between the robot and human, the suppression 
of collision/ contact forces has been investigated using the hybrid joints, and the control laws for 
the restoration of desired position have also been developed. This control laws are capable of 
maneuvering the end-effector position, which is tolerant to unexpected contact forces with the 
environment. The joint configurations are re-adjusted according to the position of the end-effector 
to keep performing the given task. The mobile base is utilized to improve the flexibility of the 
system. This kind of system can be implemented easily and employed effectively in unknown 
dynamic environments such as offices, and home environments, due to the demonstrated 
capabilities of compliance behavior and collision tolerance. Validity of the proposed mechanism 
and control strategy has been verified by simulation and experimental tests. 
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1. Introduction 

Robots have been beginning to move from industrial fields such as factories, to offices, 
houses, and schools. Furthermore, a great deal of study has been performed recently on 
robots that feature functions for communicating with humans in daily life, (i.e., 
communication robots). This research has many applications such as entertainment, 
education, psychiatry, and so on (Dautenhahn et al., 2002; Druin & Hendler, 2000). On the 
other hand, some research has found that humans tend to have either extremely positive or 
extremely negative attitudes toward novel communication technologies (Joinson, 2002). If 
communication robots can be regarded as a novel communication technology, there is the 
possibility that humans will have negative attitudes or emotions toward these robots, 
regardless of whether they are pet- types or humanoid robots. Thus, it should be carefully 
investigated on how humans are mentally affected by them. 

Regarding the measurement of human mental images and impressions toward robots, there 
are plenty of published studies. Shibata et al. (2002; 2003; 2004) reported international 
research results on people's subjective evaluations of a seal-type robot they developed, called 
Palo, in several countries including Japan, the U. K, Sweden, Italy, and Korea. Their results 
revealed that there were differences in subjective evaluations of the robot among genders and 
ages, and that nationality also affected the evaluation factors. Friedman et al. (2003) 
investigated people's relationships with robotic pets by analyzing more than 6,000 postings in 
online discussion forums about one of the most advanced robotic pets currently on the retail 
market, Sony's robotic dog AIBO. Furthermore, Kahn et al. (2004) examined preschool 
children's reasoning about and behavioural interactions with AIBO. Their important 
suggestion is that people in general, and children in particular, may fall prey to accepting 
robotic pets without the moral responsibilities that real, reciprocal companionship and 
cooperation involves. In addition, Nomura et al. (2005b) reported the results of social research 
on visitors to an exhibition of communication robots, called "Robovie" (Ishiguro et al., 2001), 
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suggesting that even in Japan, younger generations do not necessarily like the robots more 
than do elder generations. These studies are focused on specific commercialized robots. 
On the other hand, some studies examined more general images independent of specific robots. 
Suzuki et al. (2002) developed a psychological scale for measuring humans' mental images toward 
robots, while Kashibuchi et al. (2002) showed by using this scale that humans' mental images 
toward robots are positioned in the middle of a one-dimensional scale, where one pole 
corresponds to humans and another pole corresponds to just physical objects. Woods & 
Dautenhahn (2005) investigated the difference in relations of robots' appearances to emotions 
toward them between children and adults, using a questionnaire-based method and photographs 
of several robots. In addition, Bartneck et al. (2005a; 2005b) reported the influences of cultural 
differences and personal experiences with robots into attitudes toward robots by using a 
psychological scale measuring negative attitudes toward robots (Nomura et al., 2006a). 
The above studies are based on social research. Furthermore, there are experimental studies 
on psychological influence to human behaviours toward robots. 

Reeves & Nass (1996) showed that humans tend to unconsciously react to machines in the 
same way as to humans, even though they recognize the fact that they are interacting with 
machines (Media Equation). Kanda et al. (2001) investigated the impression on humans of a 
humanoid robot "Robovie", mentioned above, based on a psychological experiment of 
interaction with the robot, measuring impressions with the semantic-differential method. 
Goetz et al. (2003) proposed a " matching hypothesis 77 to explore relationships between 
robots' appearances and tasks, and found that more friendly tasks match more friendly 
appearances. Kidd & Breazeal (2004) conducted some experiments to compare appearances 
between artificial agents and robots, and found that robots are more suitable than agents for 
tasks such as pointing at objects related to real spaces. Walters et al. (2005) experimentally 
investigated relations between humans' personal traits measured by some questionnaires 
and behaviours toward robots such as permitted distances between them. Furthermore, 
Nomura et al. (2006a) experimentally investigated relations between humans' negative 
attitudes toward robots and communication avoidance behaviours toward them. 
The studies mentioned above show an increase of psychological research on human- 
robot interaction. In this stream, we have focused on people's negative attitudes 
toward robots and relations to behaviours toward robots (Nomura et al., 2006a). This 
chapter provides the concept of negative attitudes toward robots and a measurement 
method for them, "Negative Attitudes toward Robots Scale (NARS)", as a 
psychological index in research on human-robot interaction. Then, it shows results of 
psychological experiments and social research by using this psychological scale. The 
former explores relationships between negative attitudes and behaviours toward 
robots in a real situation of human robot interaction, and the latter investigates 
relationships between negative attitudes toward and assumptions about robots. Finally, 
it is discussed on how people's attitudes toward robots can be altered in near future. 

2. Negative Attitudes toward Robots Scale (NARS) 

An attitude is psychologically defined as a relatively stable and enduring predisposition to behave 
or react in a certain way toward persons, objects, institutions, or issues, and the source is cultural, 
familial, and personal (Chaplin 1991). This definition of attitudes implies that they can be affected 
by cultural backgrounds and personal experiences. Moreover, the classical psychological theory 
suggests that they can be changed based on mental congruity (Osgood & Tannenbaum 1955; 
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Newcomb 1953; Heider 1958). These facts imply that attitudes toward robots can be altered by 
some factors including cultural backgrounds and personal experiences. 

As one of the tools to measure humans' psychological factors in interaction with robots, the 
Negative Attitude toward Robots Scale (NARS) was developed to determine humans' 
attitudes toward robots. Its internal consistency, factorial validity, and test-retest reliability 
have been confirmed based on Japanese respondents (Nomura et al., 2006a). 

Item No. Questionnaire Items Sub Scale 

1 I would feel uneasy if robots really had emotions. S2 

2 Something bad might happen if robots developed into living S2 
beings. 

3 I would feel relaxed talking with robots. * S3 

4 I would feel uneasy if I was given a job where I had to use robots. SI 

5 If robots had emotions, I would be able to make friends with them. S3 

6 I feel comforted being with robots that have emotions. * S3 

7 The word "robot" means nothing to me. SI 

8 I would feel nervous operating a robot in front of other people. SI 

9 I would hate the idea that robots or artificial intelligences were SI 
making judgments about things. 

10 I would feel very nervous just standing in front of a robot. SI 

11 I feel that if I depend on robots too much, something bad might S2 
happen. 

12 I would feel paranoid talking with a robot. SI 

13 I am concerned that robots would be a bad influence on children. S2 

14 I feel that in the future society will be dominated by robots. S2 
(^Reverse Item) 

Table 1. Questionnaire Items in NARS and Subscales that the Items Included. 

This scale consists of fourteen questionnaire items. Table 1 shows the English version of the NARS, 
which was translated using back-translation. These items are classified into three subscales, SI: 
"Negative Attitude toward Situations of Interaction with Robots" (six items), S2: "Negative 
Attitude toward Social Influence of Robots" (five items), and S3: "Negative Attitude toward 
Emotions in Interaction with Robots" (three items). The number of grades for each item is five (1: 1 
strongly disagree, 2: I disagree, 3: Undecided, 4: I agree, 5: I strongly agree), and an individual's 
score on each subscale is calculated by summing the scores of all the items included in the subscale, 
with the reverse of scores in some items. Thus, the minimum and maximum scores are 6 and 30 in 
SI, 5 and 25 in S2, and 3 and 15 in S3, respectively. 

3. Negative Attitudes toward Robots in Human-Robot Interaction 
Experiments 

By using the NARS, we designed and conducted experiments where subjects interacted 
with a humanoid type communication robot "Robovie," which is being developed as a 
platform for research on the possibility of communication robots (Ishiguro et al., 2001) 
to investigate the influence of their negative attitudes toward robots into their 
behaviors toward them. 
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The results of the previous experiment conducted in the summer of 2003 (Nomura et 
al., 2006a) suggested a possibility that negative attitudes toward robots affected 
human behaviors toward communication robots. Moreover, it suggested a possibility 
that there were gender differences in negative attitudes toward robots, and that there 
were also gender differences in relations between negative attitudes and behaviors 
toward robots. Furthermore, it also suggested a possibility that individuals' 
experiences of real robots influence the relations between negative attitudes and 
behaviors toward robots. Considering these results, we designed and conducted a 
new experiment at December, 2005. 

This section provides with some results of the experiment, in particular, relationships 
between communication avoidance behaviors and negative attitudes toward robots based 
on regression analysis. 

3.1 The Robot Used in the Experiments 

As shown in Fig. 1, Robovie is a robot that has a human-like appearance and is 
designed for communication with humans (Ishiguro et al., 2001). It stands 120 cm tall, 
its diameter is 40 cm, and it weighs about 40 kg. The robot has two arms (4X2 DOF), a 
head (3 DOF), two eyes (2X2 DOF for gaze control), and a mobile platform (two 
driving wheels and one free wheel). 

The robot has various sensors, including skin sensors covering the whole body, 10 tactile 
sensors located around the mobile platform, an omni-directional vision sensor, two 
microphones to listen to human voices, and 24 ultra-sonic sensors for detecting obstacles. It 
carries a Pentium III PC on board for processing sensory data and generating gestures. The 
operating system is Linux. 







Fig 1. Robovie. 
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Fig. 2. Overview of the Room Where the Experiment Were Executed (A View from above). 

3.2 Experimental Procedures 

Our experiments on human-robot interaction were executed in the room shown in Fig 2. 
Robovie programmed in advance was prepared for interaction with subjects in the room, 
and each subject communicated with it for a few minutes alone. The procedures used in one 
session of the experiments are shown as follows: 

1. Before entering the experiment room shown in Fig 2, the subjects responded to the 
following questionnaire items: 1: sex, 2: age, 3: the NARS, 4: State-Trait Anxiety 
Inventory (STAI) (Spielberger et al, 1970). 

2. Just before entering the room, they were instructed to talk toward Robovie just after 
entering the room. 

3. The subject entered the room alone. Then, he/she moved in front of the robot. 

4. After he/she talked to Robovie, or a constant time (30 seconds) passed, Robovie 
uttered a sentence to stimulate his/her self-expression ("Tell me one thing that 
recently happened on you.") 

5. After he/she replied to the utterance of Robovie, or a constant time (30 seconds) passed, 
Robovie uttered a sentence to stimulate his/her physical contact to it ("Touch me"). 

6. After he/she touched the body of Robovie, or a constant time (30 seconds) passed, 
the session finished. 
STAI (Spielberger et al., 1970) was used to measure the subjects' general anxiety before the 
experiment and analyze its relations to the negative attitudes toward robots. The emotion of 
anxiety is generally classified into two categories: state and trait anxiety. Trait anxiety is a 
trend of anxiety as a characteristic stable in individuals whereas state anxiety is an anxiety 
transiently evoked in specific situations and changed depending on the situation and time. 
STAI consists of twenty items for measuring state anxiety (STAI-S) and twenty items for 
measuring trait anxiety (STAI-T). In this experiment, only STAI-S was used. 
Behaviours of the subjects, including their utterances, were recorded using two digital video 
cameras as shown in Fig 2. One of the cameras was installed into Robovie 7 s eyes. Then, the 
following items related to their behaviours were extracted from the video data: 
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• The distance from the subjects to Robovie when they first stood in front of the robot 
after entering the room (D) 

• The time elapsed until the subjects talked to Robovie after entering the room (Ul) 

• The time elapsed until the subjects replied to Robovie after it uttered to stimulate 
their self-expression (U2) 

• The time elapsed until the subjects touched the robot's body after it uttered to 
stimulate the subjects' physical contact with it (T) 

Moreover, the contents of the subjects' utterances in the above step 5, that is, their replies to 
stimulation from the robot for their self-expression, were classified into two categories: 
utterances including the subjects' emotions and utterances not including those (all the 
subjects had some utterances in this step). This classification was executed by two persons, 
and if there was a difference between classification results of the two persons they discussed 
and integrated their classification results. 

3.3 Results 

Thirty-eight Japanese university students were asked to participate in the experiment as 
subjects (male: 22, female: 16), and the mean age of these subjects was 21.3. Table 2 shows 
the means and standard deviations of their NARS scores and behaviour indices, t- tests for 
these values revealed that there was no difference on these values between the male and 
female subjects, except for the statistically significant trend in the S2 score of the NARS. 

N Mean SD t p 

-.171 .865 

1.994 .054 

.711 .482 

-.519 .607 

.296 .769 

-.451 .655 

-.804 .429 

Table 2. Means and Standard Deviations of NARS Scores and Behaviour Indices, and 
Results of £-Tests. 

First, linear regression analysis was conducted to investigate relations between the NARS 
scores and behaviour indices. In this analysis, the independent variables were the NARS 
scores and the dependent variables were the behaviour indices. The previous experiments 
suggested gender differences on relations between these independent and dependent 
variables (Nomura et al., 2006a). Thus, the analysis was conducted for the whole samples, 
only the male samples, and only the female samples, respectively. 

Table 3 shows the statistically significant regression models in the analysis. It was revealed that the 
time elapsed until the subjects talked to Robovie after entering the room (Ul) and the time elapsed 
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until the subjects touched the robot's body after it uttered to stimulate the subjects' physical contact 
with it (T) were influenced by the negative attitude toward situations of interaction with robots 
(SI) and the negative attitude toward social influence of robots (S2), respectively. In the whole 
samples, SI positively influenced into Ul and S2 negatively influenced into T. 
The analysis for the limited samples of each gender suggested a gender difference on 
relations between the negative attitudes and behaviours toward robots. In the male samples, 
Ul and T were influenced by SI and S2 respectively, in the same way as the case of the 
whole samples. Moreover, the negative attitude toward emotions in interaction with robots 
(S3) also positively influenced into T. On the other hand, this trend did not appear in the 
female samples. In addition, S3 positively influenced into the time elapsed until the subjects 
replied to Robovie after it uttered to stimulate their self-expression (U2) only in the female 
samples. Furthermore, the values of R 2 showed that the regression models for the limited 
samples of each gender had the higher goodness-of-fit than those for the whole samples. 
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NARS: SI 
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.650 
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.077 
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3.406 
-1.034 

.385 
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Male Samples 
Dependent Variable: T 
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NARS: S2 
NARS: S3 

Ri (N = 15) 


-.625 
.541 

.322 


-2.617 
2.263 


.023 
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Female Samples 
Dependent Variable: U2 


P 


t 


V 


NARS: S3 

Ri (N = 15) 


.526 

.222 


2.232 


.044 



Table 3. Statistically Significant Regression Models between NARS Scores and Behaviour Indices. 

Second, two-way ANOVAs for the NARS scores were conducted to investigate their 
relations to the contents of the subjects' utterances toward the robot and gender. Table 4 
shows the results of these ANOVAs. It was found that there were statistically significant 
trends of the utterance contents in SI and gender in S2. There was no interaction effect in all 
the scores. Important is the trend that the subjects whose utterances toward the robots 
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included their emotional contents had higher negative attitudes toward interaction with 
robots than those whose utterances did not include emotional contents. 
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NARS: SI 


.003 


.956 


3.884 


.057 


.641 


.429 


NARS: S2 


3.670 


.064 


.003 


.959 


.011 


.916 


NARS: S3 


.527 


.473 


2.845 


.101 


.032 


.859 



Table 4. Results of ANOVAs with Gender and Contents of Subjects' Utterances toward the 
Robot (EU: Subject Group Having Utterances Including their Emotions, NU: That Having 
Utterances Not Including their Emotions). 

Third, Pearson's correlation coefficients r between the NARS scores and STAI-S were calculated 
to investigate relations between the negative attitudes toward robots and general anxiety of the 
subjects just before the experiment. Table 5 shows these correlation coefficients. There was a 
medium level of correlation between STAI-S and SI. Moreover, there was a medium level of 
correlation between S2 and S3. Although there was no trend of gender difference in these 
correlations, it was found in the correlation between SI and S3; there was a medium level of 
correlation in the female samples although there was no such correlation in the male samples. 
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.024 
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.676** 


.175 
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(tp < .1, y < .05, y < .01, *y < .ooi) 

Table 5. Pearson's Correlation Coefficients r between NARS and STAI-S. 



3.4 Discussion 

The experiment was conducted only for the Japanese subjects. Thus, we should note that 
there is a limit for generalizing the results. 

The above results of regression analysis imply that negative attitudes toward robots are 
related to concrete behaviors toward robots, such as time spent to talk with and touch to 
them. This implies that the NARS has predictive validity for communication avoidance 
behaviors toward robots. In the previous experiment, no regression models were statistically 
significant for the same behavior indices (Nomura et al., 2006a). This may be caused by the 
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differences of the room layout, the detailed experimental procedures, and widespread of 
communication robots from the previous experiment. 

The results of regression analysis also imply the gender difference of relationships between 
negative attitudes and behaviors toward robots. In the samples of the experiment, there was 
almost no gender difference of the NARS scores and behavior indices. However, the 
experimental results imply that males having high negative attitude toward interaction with 
robots tend to avoid talking with them, and those having low negative attitude toward 
social influence of and high negative attitude toward emotions in interaction with robots 
tend to avoid touching to them. On the other hand, the results imply that females do not 
have this trend, and those having high negative attitude toward emotions in interaction 
with robots tend to avoid their self-expression toward them. 

Moreover, the correlation coefficients between SI and S3 imply the gender difference of 
relationships between attitudes toward interaction with robots and emotions in interaction 
with robots. In other words, females may have psychologically stronger connection between 
interaction itself and emotional subjects in interaction with robots than males. 
The results of ANOVAs imply the possibility that persons having high negative attitude toward 
interaction with robots tend to talk with them about contents related to their emotions. Our 
prediction before the experiment was the converse, that is, persons having high negative attitude 
toward interaction with robots avoid talking with them about contents related to their emotions, 
since avoidance of self-expression is one of examples of communication avoidance behaviours and 
the results of the previous experiment partly supported it even in human-robot interaction 
(Nomura et. al., 2006a). Although we have not found the cause, we can consider an interpretation: 
persons having high negative attitude toward interaction with robots have high anxiety toward 
communication with robots, high anxiety toward communication lead them to their desire to get 
opportunities of communication, and conversely they talk with robots about things related to 
themselves to complement a blank in communication. 

The correlation coefficients between SI and STAI-S suggest a strong connection between 
general anxiety and negative attitudes toward interaction with robots in the situation just 
before human-robot interaction. The above interpretation may be verified by measurement 
of anxiety toward robots (Nomura et al., 2004; 2006b). 

4. Negative Attitudes toward Robots in Social Research 

This section provides with some results of social research by using the NARS. 
There are some existing studies on social research using the NARS. Bartneck et al., (2005a; 
2005b) suggested that cultures may influence attitudes toward robots. Moreover, Nomura et 
al., (2006a) also suggested the influence of personal experiences with robots on attitudes 
toward them, such as individuals' experiences on really acting robots. However, these 
studies lack perspective on what types and tasks of robots people assume. 
It is considered that attitudes toward robots can more directly be influenced by assumptions 
about robots than by cultures and personal experiences, although these assumptions can be 
affected by cultures and personal experiences. Thus, we should focus on not only attitudes 
toward robots but also assumptions about robots. These assumptions can be influenced by 
cultural situations such as media, and their distribution can affect that of attitudes toward 
robots. Of course, cultural differences can produce differences in attitudes toward one 
specific type of robot such as humanoids, as Kaplan (2004) mentioned. Thus, we can 
consider several types of differences in attitude toward robots as follows: 
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1. Differences in attitudes between different assumptions about robots in one culture. 

2. Differences in attitudes toward one specific type of robot between different cultures. 

3. Differences in assumptions about robots and their relationship to attitudes toward 
robots between different cultures. 

As mentioned above, Bartneck et al. (2005a; 2005b) suggested the possibility of there being 
cultural differences in attitudes toward robots. This section deals with the first issue in the 
above list. As a concrete subject, we administered a social research to investigate the 
differences in attitudes between assumptions about robots in one culture, Japan. 

4.1 Method 

The social research was administered from November 2005 to March 2006. The 
participants were Japanese university and special training school students. The 
Japanese version of the NARS was administered during lecture time. Participation by 
the respondents was voluntary. 

The face sheet used in administering this survey included items that asked respondents to answer 
which type of robots they assumed and which tasks they assumed the selected robots do. The 
choices for the former item were: 1: human-size humanoids, 2: small-size humanoids, 3: big active 
robots, 4: animal-type robots, 5: stationary machines, 6: arm manipulators, and 7: others. The 
choices for the latter item were: 1: housework, 2: office work, 3: public service such as education, 4: 
medical or welfare service, 5: construction or assembling tasks, 6: guard or battle, 7: tasks in places 
hard for humans to go or hazardous locations such as the space and the deep sea, 8: the service 
trade, 9: communication partners or playmates, 10: amusement, 11: others. These choices were 
determined based on the pilot study by Nomura et al. (2005a). 

In addition to the above face sheet and the NARS, two psychological scales were 
administered to investigate relationships between attitudes toward robots and personal 
traits. One is the STAI. In this social research, both STAI-S and STAI-T were used. The other 
scale is the Personal Report of Communication Apprehension (PRCA-24) (Pribyl et al., 1998). 
PRCA-24 measures communication apprehension in four contexts: public speaking, 
meetings, small-group discussion, and dyads. Each context corresponds to six items. In this 
administration, only six items corresponding to dyads were used to investigate their 
relationships with the NARS subscales directly related to interaction with robots (SI and S3). 

4.2 Results 

A total of 400 people (male: 197; female: 199; unknown: 4; mean age: 21.4) participated in the 
research. For the 374 samples that had no missing item in the NARS, Cronbach's a denoting 
reliability were 0.756 for SI, 0.647 for S2, and 0.735 for S3 respectively. The sample data were 
analyzed in the following three ways. 

4.2.1 Relations between Assumptions about Robot Types and Tasks 

First, we calculated how many respondents selected each robot type and task with respect to 
the assumptions about robots. Then, to find relations between specific assumptions about 
types and tasks, cp-coefficients were calculated to reveal the extent of relationships between 
the assumption choices. In addition, we performed statistical tests with Fisher's method on 
selection for pairs of choices to investigate the statistical significance of these cp-coefficients 
based on the independence among these choices (for example, to investigate the correlation 
between "small-size humanoids" and "amusement,", one 2X2 cross table consisting of 
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selection/ no-selection of " small-size humanoids" and " amusement " was made, then the cp- 
coefficients was calculated and a test was done for this cross table). 
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Robot Type: 1: human-size humanoids, 2: small-size humanoids, 3: big active robots, 4: 
animal-type robots, 5: stationary machines, 6: arm manipulators, 7: others 

Task Type: 1: housework, 2: office work, 3: public service such as education, 4: medical or 

welfare service, 5: construction or assembling tasks, 6: guard or battle, 7: tasks in 
places hard for humans to go or hazardous locations such as the space and the 
deep sea, 8: the service trade, 9: communication partners or playmates, 10: 
amusement, 11: others 

(*p < .05, **p < .01, ***p < .001) 

Table 6. The Number of Respondents Who Selected Each Robot Type and Task, and 
Correlations. 

Table 6 shows the number of respondents who selected each robot type and task, and the 
correlations between robot types and tasks. Regarding assumptions about robot type, about 
50% of the respondents selected "human-size humanoids/' The humanoid type, including 
small-size ones, was selected by about 70% of the respondents, while the selection rate for 
"animal-type robots' 7 was about 7%. The respondents who selected "others" tended to 
mention concrete names of some robots appearing in the media, such as "Doraemon" and 
"Asimo" in their written answers. 
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Regarding assumptions about robot tasks, there was no bias of respondents toward a 
specific task; the selection rates for ''housework/ 7 " communication partner or playmates/ 7 
and // amusement ,/ varied from 16% and 22%. The selection rates for " guard or battle," and 
"tasks in places hard for humans to go or hazardous locations" were about 12"-13%. Few 
respondents selected "public service such as education." The respondents who selected 
"others" tended not to mention concrete tasks in their written answers. 

Regarding relations between robot type and task, there was a moderate level of positive 
correlation between "big active robots" and "guard or battle," between "animal-type robots" and 
"communication partners or playmates, "between "arm manipulators" and "construction or 
assembling tasks," and between "others" and "others." Moreover, there was a low level of positive 
correlation between "human-size humanoids" and "housework," between "human-size 
humanoids" and "the service trade," between "small-size humanoids" and "amusement," 
between "stationary machines" and "public service such as education," between "stationary 
machines" and "construction or assembling tasks," and "others" and "communication partners or 
playmates." In addition, there was a low level of negative correlation between "human-size 
humanoids" and "communication partners or playmates," between "small-size humanoids" and 
"construction or assembling tasks," between "small-size humanoids" and "guard or battle," 
between "big active robots" and "housework," between "big active robots" and "communication 
partners or playmates," and between "animal-type robots" and "housework." 

4.2.2 Relations of Gender and Assumptions about Robots with Attitudes toward Robots 

Second, to investigate the relations between attitudes toward and assumptions about robots, 
two-way ANOVAs were executed with the independent variables of gender and robot type, 
and variables of gender and robot task, respectively. In this analysis, the subgroups of 
respondents who selected "stationary machines," "arm manipulators," and "others" were 
integrated into one subgroup due to their small numbers of respondents and correlations 
with robot tasks. Furthermore, the subgroups of respondents who selected "office work," 
"public service such as education," "medical or welfare service," "the service trade," and 
"others" were integrated into one subgroup, due to their small numbers of respondents, 
correlations with robot type, and the similarity of their contents. 

Table 7 shows the means and standard deviations of the scores of the NARS subscales based 
on gender and robot type subgroups, and the results of the two-way ANOVAs for the 
NARS subscale scores. Table 8 shows the means and standard deviations of the scores of the 
NARS subscales based on gender and robot task subgroups, and the results of the two-way 
ANOVAs for the NARS subscale scores. 

For the ANOVA of gender and robot type, there were statistically significant effects regarding 
both gender and robot type on the scores of SI and S3, although there were no interaction effects. 
It was revealed that the female respondents had more pronounced negative attitudes toward 
situations of interaction with robots and lower negative attitudes toward emotions in interaction 
with robots, than the male respondents. Moreover, the post-hoc tests with Tukey's method 
revealed with a 5% significance level that the respondents in the subgroups of "small-size 
humanoids" and "big active robots" had stronger negative attitudes toward situations of 
interaction with robots than those in the integrated subgroup consisting of "stationary objects," 
"arm manipulators," and "others." In addition, they also revealed that the respondents in the 
subgroup of "small-size humanoids" had more pronounced negative attitudes toward emotions 
in interaction with robots than those in the subgroups of "human-size humanoids" and "animal- 
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type robots/' There were no statistically significant effects of gender, robot type, or interaction in 
S2, negative attitude toward the social influence of robots. 









NARS: SI 


NARS: S2 


NARS: S3 






N 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Male 


1 


92 


12.5 


3.7 


16.0 


3.6 


9.5 


2.7 




2 


27 


12.3 


4.2 


16.3 


2.8 


11.1 


2.2 




3 


34 


13.6 


5.3 


16.2 


4.6 


10.3 


2.9 




4 


13 


12.7 


4.6 


15.6 


3.9 


9.6 


1.6 




5+6+7 


16 


9.9 


5.1 


14.8 


5.4 


11.4 


1.9 


Female 


1 


89 


13.7 


3.8 


16.2 


3.6 


9.2 


2.6 




2 


46 


14.8 


3.7 


17.1 


4.0 


10.5 


2.6 




3 


21 


15.2 


4.9 


16.4 


3.8 


9.6 


2.6 




4 


13 


14.8 


2.5 


14.8 


3.4 


8.1 


2.3 




5+6+7 


17 


12.5 


3.9 


16.1 


4.9 


9.1 


2.9 




Gender 
F p 


Robot Type 
F ^ p 


Interaction 
F p 


Post-hoc 


NARS: SI 


13.910 


.000 


3.386 


.010 


.498 


.738 


2,3 


> 5+6+7 


NARS: S2 


.420 


.517 


1.063 


.375 


.361 


.836 






NARS: S3 


10.237 


.001 


5.406 


.000 


1.285 


.276 


2>1,4 



Table 7. Means and Standard Deviations of the Scores of the NARS Subscales based on 
Gender and Robot Type Subgroups, and Results of the two-way ANOVAs. 









NARS: SI 


NARS: S2 


NARS: S3 






N 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Male 


1 


30 


12.4 


3.7 


15.7 


3.5 


9.9 


2.4 




5 


12 


11.5 


4.0 


16.3 


2.9 


10.7 


2.1 




6 


27 


11.4 


4.5 


15.6 


4.2 


10.3 


2.9 




7 


28 


13.9 


4.5 


16.8 


3.9 


9.8 


2.9 




9 


22 


11.6 


4.1 


14.3 


3.9 


9.2 


2.1 




10 


28 


13.1 


4.0 


16.5 


3.9 


10.4 


2.5 




Others 


31 


12.9 


5.0 


16.1 


4.1 


10.0 


2.9 


Female 


1 


33 


14.7 


4.7 


16.3 


3.3 


9.3 


2.6 




5 


5 


13.6 


7.5 


18.8 


4.1 


10.4 


0.9 




6 


16 


12.8 


4.2 


16.6 


3.2 


10.3 


2.9 




7 


19 


15.7 


3.0 


18.4 


4.2 


10.3 


2.2 




9 


59 


14.3 


3.2 


15.4 


3.8 


8.9 


3.0 




10 


34 


13.6 


3.4 


16.4 


3.9 


9.6 


2.7 




Others 


18 


12.6 


4.1 


15.7 


3.7 


9.3 


1.8 




Gender 


Robot Task 


Interaction 


Post-hoc 




F 


V 


F 


V 


F 


V 






NARS: SI 


8.460 


0.004 


1.899 


0.080 


0.875 


0.513 






NARS: S2 


3.576 


0.059 


2.774 


0.012 


0.543 


0.775 


7>9 




NARS: S3 


0.914 


0.340 


1.421 


0.206 


0.360 


0.904 







Table 8. Means and Standard Deviations of the Scores of the NARS Subscales based on 
Gender and Robot Task Subgroups, and Results of the two-way ANOVAs. 
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For the ANOVA of gender and robot task, there was a statistically significant effect of robot 
task on the scores of S2. The post-hoc tests with Tukey's method revealed with a 5% 
significance level that the respondents in the subgroup of " tasks in places hard for humans 
to go or hazardous locations" had stronger negative attitudes toward the social influence of 
robots than those in the subgroup of "communication partner or playmates." Moreover, 
there were statistically significant trends regarding robot task in SI and gender in S2. 

4.2.3 Correlations between NARS, STAI, PRCA-24 

Third, to investigate the relations between attitudes toward robots, general anxiety, and 
communication apprehension, we calculated Pearson's correlation coefficients r between the 
NARS subscales, STAI, and PRCA-24. Since there is a possibility of gender difference with respect 
to anxiety and communication apprehension, this calculation was done for each gender subgroup. 
Table 9 displays these coefficients. The table reveals that there was a moderate level of 
correlations among SI, STAI-S, and STAI-T for the female respondents, although that for the 
male respondents was low. Moreover, it also reveals that there was a low level of correlation 
among SI, S2, and PRCA-24, although there was no such correlation for the male 
respondents. There was no correlation among S3, STAI-S, and PRCA-24. 









NARS: SI 


NARS: S2 


STAI-S 


STAI-T 


PRCA-24 


NARS: SI 


Whole 


r 






0.257* 


0.263* 


0.080 






N 






355 


356 


338 




Male 


r 






0.181* 


0.185* 


0.008 






N 






171 


171 


160 




Female 


r 






0.333* 


0.282* 


0.160* 






N 






182 


183 


176 


NARS: S2 


Whole 


r 


0.401** 




0.082 


0.121* 


0.110* 






N 


371 




355 


356 


338 




Male 


r 


0.378** 




0.043 


0.092 


0.041 






N 


182 




171 


171 


160 




Female 


r 


0.452** 




0.106 


0.136 


0.166* 






N 


187 




182 


183 


176 


NARS: S3 


Whole 


r 


0.125* 


0.360** 


-0.004 


-0.145** 


0.044 






N 


371 


371 


355 


356 


338 




Male 


r 


0.130 


0.274** 


-0.076 


-0.135 


-0.020 






N 


182 


182 


171 


171 


160 




Female 


r 


0.180* 


0.469** 


0.102 


-0.092 


0.138 






N 


187 


187 


182 


183 


176 



Table 9. Pearson's Correlation Coefficients r between NARS Subscales, STAI, and PRCA-24. 



4.3 Discussion 

In the same way as the experiment presented in the previous section, the results of the social 
research are based on Japanese respondents, that is, on one specific culture. We carefully discuss 
the results' implications, separating those limited to Japanese culture from results that may be 
generalized to other cultures. The latter issue are related to alternation of attitudes toward robots. 
Here, we discuss the former issue since the latter are discussed in the next section. 
The results for assumptions about robot type suggest a bias of respondents toward humanoid- 
type robots. On the other hand, cp-coefficients between this type and assumptions about robot 
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tasks suggest that "humanoid robots'' were not strongly related to specific tasks. Moreover, the 
correlations between "big active robots" and "guard or battle," between "animals" and 
"communication partners or playmates," and between "arm manipulators" and "construction or 
assembling tasks" suggest conservative images of robots that have been reconstructed via the 
media. This trend is similar to the results by Nomura et al. (2005a). These suggestions imply that 
more Japanese people are more biased toward humanoid-type robots than other types, but are 
unclear about what tasks this type of robot does. 

The results of the ANOVAs for the NARS subscale scores suggest that the respondents assuming 
"small-size humanoids" and "big active robots" had more pronounced negative attitudes toward 
interaction with robots than those assuming "stationary machines," "arm manipulators," and 
"others." We assume that the robot types "stationary machines," "arm manipulators," and 
"others" lead people to have conservative images of robots, such as being big computer, industrial 
robots, and animated robots. Thus, this suggestion implies that novel types of robots or robots 
related to battle evoke negative attitudes toward human interaction with robots. 
Moreover, the results of the ANOVAs suggest that the respondents assuming "small-size 
humanoids" had stronger negative attitudes toward emotions in interaction with robots 
than those assuming "human-size humanoids" and "animal-type robots." This implies that 
emotional reactions toward robots are different between robot types, depending on 
interaction effects between design and size. 

In addition, the results of the ANOVAs suggest that the respondents assuming "tasks in 
places hard for humans to go or hazardous locations" were more negative toward the social 
influence of robots than those assuming "communication partner or playmates." We 
estimate that the image of danger in the former task assumption evoked negative attitudes 
toward the social influence of robots performing such tasks. 

The ANOVA results also suggest that the female respondents had more pronounced negative 
attitudes toward interaction with robots and less negative attitudes toward emotions in 
interaction with robots than did the male respondents. Furthermore, the correlation coefficients 
among the NARS, STAI, and PRCA-24 suggest that there is also a gender difference regarding 
relations between negative attitudes toward robots and personal traits related to anxiety. This 
suggestion implies that there is a gender difference in attitudes toward robots, depending on 
which factor we focus on in interaction with robots, and gender-based difference in their 
relations to some personal traits. 

On the other hand, the correlation coefficients between the NARS subscales and PRCA-24 
suggest that there is a low level of correlation between attitudes toward robots and 
communication apprehension as a personal trait related to interaction. Moreover, the correlation 
coefficients between the NARS subscales, and those between the NARS SI and STAI-S were 
much different from those in the experiment presented in the previous section. These imply that 
attitudes toward robots are altered depending on situations such as daily life and real interaction 
with robots, and these attitudes are not directly connected to personal communication traits at 
the present daily life situation where robots such as humanoids are not yet widespread and 
images of their tasks are not yet fixed. 

5. Implications 

Although the implications outlined in the previous sections should be limited to Japanese 
culture in a strict sense, we can extend our discussion to other cultures to some extent. This 
section discusses on how people's attitudes toward robots can be altered in near future, 
based on the implications in the previous section. 
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The results of the experiment and social research imply that attitudes toward robots 
are related to concrete behaviours toward them, but they are altered depending on 
situations. The short-term alternation is dependent on individual situations such as 
real experiences of human-robot interaction, and the long-term alternation is 
influenced by cultural trends. 

While bias toward and relations between certain assumptions about robots may be specific 
to each culture, what is important is that attitudes toward robots may depend on specific 
assumptions dominant in a given culture. In other words, if the dominant assumptions 
about robots are changed, the whole trend of attitudes toward robots can be altered in that 
culture. This may be caused by commercialization of really acting robots and media 
information about them. People may have negative attitudes toward robots unfamiliar to 
their culture, but as information about robots spreads, their assumptions may change and 
attitudes toward them may also alter. 

An important problem is that gender differences may affect the alteration of attitudes 
toward robots. If there are currently gender differences in attitudes toward robots and 
their relations to some personal traits in a culture, these differences may affect the 
nature of attitude change toward robots; for example, males may develop more positive 
attitudes toward humanoids whereas females in the same culture may come to have 
more negative attitudes toward them. Of course, the trend of attitude change may 
depend on the cultures. 

Furthermore, it is not clear which personal trait affects attitude change toward robots. 
Although currently there may be no strong relation between attitudes toward robots and 
communication apprehension in a given culture, as implied in the previous section, the 
increasing ability of robots, in particular those related to communication, can change 
assumptions about robots, and as a result can change the connection between attitudes 
toward robots and personal traits related to communication. 

6. Conclusion 

This chapter provided the concept of negative attitudes toward robots and a measurement 
method for them, " Negative Attitudes toward Robots Scale (NARS)", as a psychological 
index in research on human-robot interaction. Then, it showed the results of some 
experiments and social research by using this psychological scale. 

The results implicated by the NARS show the efficiency of this scale in both human-robot 
interaction experiments and social research. They revealed that attitudes toward robots are 
related to behaviours toward them, these attitudes are different depending on situations and 
assumptions about robots, and there may be gender differences associated with them. 
These implications may contribute to design issues of communication robots and marketing 
research of robots in daily-life applications, in particular, the field in which communication 
capacity is critical, such as welfare. 

Furthermore, it was discussed on how people's attitudes toward robots have been 
determined and can be altered in near future. Alternation of attitudes toward robots is an 
important problem in considering development of communication robots in near future. To 
investigate this problem further, we should explore the psychological relationships between 
assumptions about robots, attitudes toward robots, and concrete emotions evoked in 
interaction with robots in more detail (Nomura et al. 2004; 2006b). Moreover, we need to 
investigate cultural differences in assumptions about robots. 
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Abstract - It is well known that in social interactions, implicit communication between the 
communicators plays a significant role. It would be immensely useful to have a robotic system that 
is capable of such implicit communication with the operator and can modify its behavior if 
required. This paper presents a framework for human-robot interaction in which the operator's 
physiological signals were analyzed to infer his/her probable anxiety level and robot behavior was 
adapted as a function of the operator affective state. Peripheral physiological signals were measured 
through wearable biofeedback sensors and a control architecture inspired by Riley's original 
information-flow model was developed to implement such human-robot interaction. The target 
affective state chosen in this work was anxiety. The results from affect-elicitation tasks for human 
participants showed that it is possible to detect anxiety through physiological sensing in real-time. 
A robotic experiment was also conducted to demonstrate that the presented control architecture 
allowed the robot to adapt its behavior based on operator anxiety level. 

Keywords: human-robot interaction, implicit communication, physiological sensing, affective 
computing, anxiety 

1. Introduction 

There has been a steady progress in the field of intelligent and interactive robotics over the 
last two decades ushering in a new era of personal and service robots. The World Robotics 
2005 survey [1] reports that over 1,000,000 household robots were in use, a number that is 
anticipated to exceed several million in the next few years. As robots and people begin to co- 
exist and cooperatively share a variety of tasks, "natural" human-robot interaction that 
resembles human-human interaction is becoming increasingly important. 
Reeves and Nass [1] have already shown that people's interactions with computers, TV and 
similar machines/ media are fundamentally social and natural, just like interactions in real 
life. Human interactions are characterized by explicit as well as implicit channels of 
communication. While the explicit channel transmits overt messages, the implicit one 
transmits hidden messages about the communicator (his/her intention, attitude and 
like/ dislike). Ensuring sensitivity to the other party's emotions or sensibility is one of the 
key tasks associated with the second, implicit channel [3]. Picard in [5] states that "emotions 
play an essential role in rational decision-making, perception, learning, and a variety of 
other cognitive functions ". Therefore, endowing robots with a degree of emotional 
intelligence should permit more meaningful and natural human-robot interaction. 
The potential applications of robots that can detect a person's affective states and interact 
with him/her based on such perception are varied and numerous. Whether it is the domain 
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of personal home aids that assist in cleaning and transportation, toy robots that engage and 
entertain kids, professional service robots that act as assistants in offices, hospitals, and 
museums, or the search, rescue and surveillance robots that accompany soldiers and 
firefighters - this novel aspect of human-robot interaction will impact them all. 
For a robot to be emotionally intelligent it should clearly have a two-fold capability-the ability to 
display its own emotions and the ability to understand human emotions and motivations (also 
called referred to as affective states). The focus of our work is to address the later capability, i.e., to 
endow a robot with the ability to recognize human affective states. There are several works that 
focus on making robot display emotions just like human beings - usually by using facial 
expressions and speech. Some prominent examples of such robots are -Pong robot developed by 
the IBM group [6], Kismet and Leonardo developed in MIT [6], and ATR's (Japan) Robovie-IIS [8]. 
Our work is complementary to this research. Fong et al. in their survey [9] provide details of many 
existing socially interactive robots that have been developed as personal robots, entertainment 
toys, therapy assistants and to serve as test beds to validate social development theories. 
There are several modalities such as facial expression, vocal intonation, gestures, postures, 
and physiology that can be utilized to determine the underlying emotion of a person 
interacting with the robot. A rich literature exists in computer vision for automatic facial 
expression recognition [10]. However, integration of such system with robots to permit real- 
time human emotion recognition and robot reaction have been very few. Bartlett et al. have 
developed a real-time facial recognition system that has been deployed on robots such as 
Aibo and Robovie [11]. Gestures and postures recognition for human-robot interaction is a 
relatively new area of research, which includes vision-based interfaces to instruct mobile 
robots via arm gestures [12]. Most of these approaches, however, recognize only static pose 
gestures. The interpretation of user emotions from gestures is a much more involved task 
and such work in the context of human-robot interaction is not known. On the other hand, 
vocal intonation is probably the most understood and valid area of nonverbal 
communication. Vocal intonation or the tone of our voice can effectively measure the 
affective (emotional) content of speech. The effects of emotion in speech tend to alter the 
pitch, timing, voice quality, and articulation of the speech signal [13] and reliable acoustic 
features can be extracted from speech that vary with the speaker's affective state [6]. 
Physiology is yet another effective and promising way of estimating the emotional state of a 
person. In psychophysiology (the branch of psychology that is concerned with the 
physiological bases of psychological processes), it is known that emotions and physiology 
(biological signals such as heart activity, muscle tension, blood pressure, skin conductance 
etc.) are closely intertwined and one influences the other. Research in affective computing, 
pioneered by Picard exploits this relationship between emotions and physiology to detect 
human affective states [14]. While concepts from psychophysiology are now being 
enthusiastically applied to human-computer interaction [15] and other domains such as 
driving [16], flying [17], and machine operation [18], the application of this technique in 
robotics domain is relatively new [19]. Our preliminary work in [20] presented concepts and 
initial experiment for a natural and intuitive human-robot interaction framework based on 
detection of human affective states from physiological signals by a robot. 
The paper is organized as follows: Section II presents the scope and rationale of the paper. 
Section III describes our proposed theoretical framework for detecting affective cues in 
human-robot interaction. This section provides the details of the physiological indices used 
for anxiety detection, use of regression tree for prediction of anxiety and the control 
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architecture for human-robot interaction. The experimental design details are presented in 
Section IV along with the results and discussion. Finally, Section V summarizes the 
contributions of the paper and provides important conclusions. 

2. Scope and Rationale of the Paper 

This paper develops an experimental framework for holistic (explicit and implicit) human- 
robot interaction by synergistically combining emerging theories and results from robotics, 
affective computing, psychology, and control theory. The idea is to build an intelligent and 
versatile robot that is affect-sensitive and capable of addressing human's affective needs. 
This paper focuses on human affect recognition by a robot based on peripheral physiological signals of 
the human obtained from wearable biofeedback sensors. While the proposed framework can be 
utilized to detect a variety of human affective states, in this paper we focus on detecting and 
recognizing anxiety. 

Anxiety was chosen as the relevant affective state to focus on for two primary reasons. 
First, anxiety plays an important role in various human-machine interaction tasks that 
can be related to task performance. Hence, detection and recognition of anxiety is 
expected to improve the understanding between humans and machines. Second, the 
correlation of anxiety with physiology is well established in psychophysiology literature 
[21] and thus provides us with an opportunity to detect it by psychophysiological 
analysis. 

Affective states have potentially observable effects over a wide range of response systems, 
including facial expressions, vocal intonation, gestures, and physiological responses (such as 
cardiovascular activity, electrodermal responses, muscle tension, respiratory rate etc.) [5]. 
However, in our work we have chosen to determine a person's underlying affective state through 
the use of physiological signals for various reasons. An attempt to examine all available types of 
observable information would be immensely complex, both theoretically and computationally. 
Also, physical expressions (facial expressions, vocal intonation) are culture, gender, and age 
dependent thus complicating their analysis. On the other hand physiological signals are usually 
involuntary and tend to represent objective data points. Thus, when juxtaposed with the self 
reports, physiological measures give a relatively unbiased indication of the affective state. 
Moreover, they offer an avenue for recognizing affect that may be less obvious to humans but 
more suitable for computers. Another important reason for choosing physiology stems from our 
aim to detect affective states of people engaged in real-life activities, such as working on their 
computers, controlling a robot, or operating a machine. In most of these cases, even if a person 
does not overtly express his/her emotion through speech, gestures or facial expression, a change 
in the physiology pattern is inevitable and detectable. Besides, there exists evidence that the 
physiological activity associated with various affective states is differentiated and systematically 
organized. There is a rich history in the Human Factors and Psychophysiology literature of 
understanding occupational stress [22], operator workload [23], mental effort [24] and other 
similar measurements based on physiological measures such as electromyogram (EMG), 
electroencephalogram (EEG), and heart rate variability (HRV). In another work, multiple 
psychophysiological measures such as HRV, EEG, and blink rates among others were employed 
to assess pilots' workload [24]. Heart period variability (HPV) has been shown to be an important 
parameter for mental workload relevant to human-computer interaction (HCI) [26]. Wilhelm and 
colleagues have also worked extensively on various physiological signals to assess stress, phobia, 
depression and other social and clinical problems [27]. 
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3. Theoretical Framework for Human-Robot Interaction Based on Affective 
Communication 

A. Physiological indices for detecting anxiety 

The physiological signals that were initially examined in our work along with the features 
derived from each signal are described in Table 1. These signals were selected because they 
can be measured non-invasively and are relatively resistant to movement artifacts. 
Additionally, measures of electrodermal activity, cardiovascular activity, and EMG activity 
of the chosen muscles have been shown to be strong indicators of anxiety [28] [29]. In 
general, it is expected that these indicators can be correlated with anxiety such that higher 
physiological activity levels can be associated with greater anxiety [30]. 
Multiple features (as shown in Table 1) were derived for each physiological measure. 
"Sym" is the power associated with the sympathetic nervous system activity of the 
heart (in the frequency band 0.04-0.15 Hz.). "Para" is the power associated with the 
heart's parasympathetic nervous system activity (in the frequency band 0.15-0.4 Hz.). 
InterBeat Interval (IBI) is the time interval in milliseconds between two "R" waves in 
the ECG waveform in millisecond. IBI ECGmean and IBI ECGstd are the mean and 
standard deviation of the IBI. Photoplethysmograph signal (PPG) measures changes in 
the volume of blood in the finger tip associated with the pulse cycle, and it provides an 
index of the relative constriction versus dilation of the blood vessels in the periphery. 
Pulse transit time (PTT) is the time it takes for the pulse pressure wave to travel from 
the heart to the periphery, and it is estimated by computing the time between systole 
at the heart (as indicated by the R-wave of the ECG) and the peak of the pulse wave 
reaching the peripheral site where PPG is being measured. Heart Sound signal 
measures sounds generated during each heartbeat. These sounds are produced by 
blood turbulence primarily due to the closing of the valves within the heart. The 
features extracted from the heart sound signal consisted of the mean and standard 
deviation of the 3rd, 4th, and 5th level coefficients of the Daubechies wavelet 
transform. Bioelectrical impedance analysis (BIA) measures the impedance or 
opposition to the flow of an electric current through the body fluids contained mainly 
in the lean and fat tissue. A common variable in recent psychophysiology research, 
pre-ejection period (PEP) derived from impedance cardiogram (ICG) and ECG 
measures the latency between the onset of electromechanical systole, and the onset of 
left-ventricular ejection and is most heavily influenced by sympathetic innervation of 
the heart [31]. Electrodermal activity consists of two main components - Tonic 
response and Phasic response. Tonic skin conductance refers to the ongoing or the 
baseline level of skin conductance in the absence of any particular discrete 
environmental events. Phasic skin conductance refers to the event related changes that 
occur, caused by a momentary increase in skin conductance (resembling a peak). The 
EMG signal from Corrugator Supercilii muscle (eyebrow) captures a person's frown 
and detects the tension in that region. It is also a valuable source of blink information 
and helps us determine the blink rate. The EMG signal from the Zygomaticus Major 
muscle captures the muscle movements while smiling. Upper Trapezius muscle 
activity measures the tension in the shoulders, one of the most common sites in the 
body for developing stress. The useful features derived from EMG activity were: mean, 
slope, standard deviation, mean frequency and median frequency. Blink movement 
could be detected from the Corrugator Supercilii activity. 



A New Approach to Implicit Human-Robot Interaction Using Affective Cues 



237 



Phvsi- -1- -L:i-.u 1 1 vaporise 


l-'eatures derived 


Lahelused 


Unit of measurement 


Cardiac activity 


Sympathetic power 


Svm 


Unit/Square Second 


himsvnii^ilhelk p> -v. er 


Para 


Unit/Square Second 


Ratio of Sym pathetic to Parasympathetic 
power 


SymPara 


Unit/Square Second 


Mean IBI 


IBIECC,,,,,,, 


Milliseconds 


Std. of 1 111 


IBI ECC„, 


Standard 1 )ev iati* 'ii ■ n« amiti 




Mean amplitude of the peak values of the 
I'l'C^inili I'livi-fldlivMii.^rann 


PPG Peak™ 


Micro Volts 


Standard deviation (Std.) of the peak 
valuesof the PPC si-jiml 


PPGPeaksLd 


Standard Deviation (no unit) 


Mean Pulse Tinns.it Time 


PTTr^n 


Milliseconds 


Heart Sound 


Mean of the 3'V\ and 5 th level 
coefficients of the Daubeclries wavelet 
transfbnn of heart sound signal 


Mean J> 
Mean d4 
Mean -_l 5 


No unit 


Standard deviation of the 3 ri ,4 lh , and 5 lh 
level coefficients of the Daubechies 
wavelet transform of heail sound si anal 


StddJ 
Stdd4 
Stdd5 


No unit 


Bioimpedance 


Me;in 1 ' io - 1 ! j ■-■-" l i- -ii I'eii-.-J 


PEP,™ 


Milliseconds 




Mean 1131 


IBI [OGw, 


Milliseconds 


Electrode mi al activity 


Mean tonic activitv level 


Tonicmui 


Micro- Siemens 


Sli ipe of tonic activity 


Toiiicikipo 


Micru-Sienii.il.-: Second 


Mean amplitude of skin conductance 
i espouse i phasic activity! 


PliasiCmiin 


Micro- Siemens 


Maxim li in amplitude of skin conductance 
ie-:p-n-:e 1 1 '1 1 M.-i ■. M-. i i - i i ■-- i 


Phasicmas 


Micro- Siemens 


Rateof phasic activity 


Phasic^ 


Response peaks/Second 


Electromyographic 
activity 


Mean of Corrugator Supercilii activity 


Cor,™ 


Micro Volts 


Std. of Coir li -s,\ tor S ll| vi\ i 1 i i ;i ■: l i v i tv 


CoiVid 


Standard 1 )ev iatii ■!! ■ n- • LuniL ■ 


M'.-|v. ■■! ' i ■■rrii^;ii--r:--Li|".r-.ilii aeli 1 . i L v 


Cordnpe 


Micro Volts Second 


\ 1 e;i n 1 nl erl-'i.;i I 1 1 He r- u 1 ■ ■ f h 1 i n k ;i ■. I i ■■ ii '- 


IBI Blink™, 


Milliseconds 


Mean am pi i tu de of hi i n k ac I i v i tv 


Amp Blink,,*.-,!! 


M ic ro Volts 


Standard dev ial \- -w ■ ■! Mink activity 


Blinknj 


Standard IX-v iatii hi i n. amiti 


Mean of Zygomatic lis Major activity 


Z>'£Wm 


Micro Volts 


Std. of Zygomatic us Major act ivih 


/v»..i 


Standard 1 h-- iatii 'ii ■ no unit! 


S In |v . ».i f /.'-. ■_:■ ■ ma 1 k li s M a i' -r ac i i \ i i ■■ 


ZYfttaa 


Micro Volts Second 


Mean of 1 pper 1 rape/ ins activity 


1 rapint.iij 


Micro Volts 


Std. i'1'l pper 1 rape z ins activity 


Trap 5 i,i 


Standard IX-v iatii 'ii i ii> • lu~i i L ■ 


Slope, of 1 pper Ha pezi lis activity 


Trapskfv 


Micro Volts Second 


Mean and Median fiHqufincy of 
Corrugator. Zygoma ticus and Trapezius 


Ztreqman 

Cfieq nMltlI1 

Tfreq mi a n etc. 


Hertz 


Tempera m re 


Mean temperature 


Tempmcii 


Degree Centi^ade 


Slope of tempera lu re 


Temp.,^.- 


Degree (. ■.nii'jru-L- Second 



Table 1. Physiological Indices. 

Mean amplitude of blink activity and mean interblink interval were also calculated from 
Corrugator EMG. The various features extracted from the physiological signals were combined in 
a multivariate manner and fed into the affect recognizer as described in the next section. Some of 
these physiological signals, either in combination or individually, have previously been used by 
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others to detect a person's affective states when deliberately expressing a given emotion or 
engaged in cognitive tasks [14]. Our approach was to detect anxiety level of humans while they 
were engaged in cognitive tasks on the computer and embed this sensing capability in a robot. To 
our knowledge, till date no work has investigated real-time modification of robot behavior based 
on operator anxiety in a biofeedback-based human-robot interaction framework. Various methods 
of extracting physiological features exist but efforts to identify the exact markers related to 
emotions, such as anger, fear, or sadness have not been successful chiefly due to person-stereotypy 
and situation-stereotypy [29]. That is, within a given context, different individuals express the 
same emotion with different characteristic response patterns (person-stereotypy). In a similar 
manner, across contexts the same individual may express the same emotion differentially, with 
different contexts causing characteristic responses (situation-stereotypy). The novelty of the 
presented affect-recognition system is that it is both individual-and context-specific in order to 
accommodate the differences encountered in emotion expression. It is expected that in the future 
with enough data and understanding, affect recognizers for a class of people can be developed. 
B. Anxiety Prediction based on Regression Tree 

In the previous research works in emotion recognition, change in emotion has been considered 
either along a continuous dimension (e.g., valence or arousal) or among discrete states. Various 
machine learning and pattern recognition methods have been applied for determining the 
underlying affective state from cues such as facial expressions, vocal intonations, and physiology. 
Fuzzy logic has been employed for emotion recognition from facial expression [33]. Fuzzy logic 
has also been used to detect anxiety from physiological signals by our research group [19] and by 
Hudlicka et al. in [17]. There are several works on emotion detection from speech based on k- 
nearest neighbors algorithm [34], linear and nonlinear regression analysis [35]. Discriminant 
analysis has also been used to detect discrete emotional states from physiological measures [36]. A 
combination of Sequential Floating Forward Search and Fisher Projection methods was presented 
in [35] to analyze affective psychological states. Neural networks have been extensively used in 
detecting facial expression [37], facial expression and voice quality [38]. The Bayesian approach to 
emotion detection is another important analysis tool that has been used successfully. In [39] a 
Bayesian classification method was employed to predict the frustration level of computer users 
based on pressure signals from mouse sensors. A Naive Bayes classifier was used to predict 
emotions based on facial expressions [40]. A Hidden Markov Model based emotion detection 
technique was investigated for emotion recognition [41]. 

In this paper we have used regression trees (also known as decision trees) to determine a person's 
affective state from a set of features derived from physiological signals. The choice of regression 
tree method emerges from our previous comparative study of four machine learning methods-K 
-Nearest Neighbor, Regression Tree, Bayesian Network and Support Vector Machine as applied 
to the domain of affect recognition [42]. The results showed that regression tree technique gave 
the second best classification accuracy - 83.5% (after Support Vector Machines that showed 
85.8% accuracy) and was most space and time efficient. Regression tree method has not been 
employed before for physiology-based affect detection and recognition. 

Regression tree learning, a frequently used inductive inference method, approximates discrete 
valued functions that adapt well to noisy data and are capable of learning disjunctive 
expressions. For the regression tree-based affect recognizer that we built, the input consisted of 
the physiological feature set and the target function consisted of the affect levels (participant's 
self-reports that represented the participant's assessment of his/her own affective state). The 
main challenge was the complex nature of the input physiological data sets. This complexity was 
primarily due to the (i) high dimensionality of the input feature space (there are currently forty 
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six features and this will increase as the number of affect detection modalities increases.), (ii) 
mixture of data types, and (iii) nonstandard data structures. Additionally, a few physiological 
data sets were noisy where the biofeedback sensors had picked up some movement artifacts. 
These data sets had to be discarded, resulting in the missing attributes. 

The steps involved in building a regression tree are shown in Figure 2. Physiological signals 
recorded from the participant engaged in PC-based task were processed to extract the input 
feature set. The participant's self -report at the end of each epoch regarding his/her affective 
states provided the target variable or the output. While creating the tree, two primary issues 
were: (i) Choosing the best attribute to split the examples at each stage, and (ii) Avoiding data 
over fitting. Many different criteria could be defined for selecting the best split at each node. In 
this work, Gini Index function was used to evaluate the goodness of all the possible split points 
along all the attributes. For a dataset D consisting of n records, each belonging to one of the m 
classes, the Gini Index can be defined as: 




Fig. 1. Creating Regression Tree. 
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where pi is the relative frequency of class i in D. If D is partitioned into two subsets Dl and D2 
based on a particular useful attribute, the index of the partitioned data Gini(D,C) can be obtained by: 

Gini{D,C) = -±Gim(I\ ) + -^ Gini{D 2 ) eq. 2 



where nl and nl are the number of examples of Dl and D2, respectively, and C is the 
splitting criterion. Here the attribute with the minimum Gini Index was chosen as the best 
attribute to split. Trees were pruned based on an optimal pruning scheme that first pruned 
branches that gave the least improvement in error cost. Pruning was performed to remove 
the redundant nodes as bigger, overfitted trees have higher misclassification rates. Thus, 
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based on the input set of physiological features described earlier, the affect recognizer 
provided a quantitative understanding of the person's affective states. 
C. Control Architecture 

As emphasized in Section 1, for a peer level human-robot interaction to mimic similar human- 
human interaction, it is essential that the robot has implicit communication in addition to explicit 
exchange of information with the human. While explicit communication allows the human and the 
robot to exchange information regarding the goals, task to be performed, the current task being 
executed and other such issues, implicit communication makes it possible for the robot to detect the 
emotional states of the human, and take necessary steps to assist the human by addressing his/her 
emotional need. There is currently no such controller that enables a robot to be responsive to an 
implicit channel of communication from the operator while it (the robot) performs its routine tasks. 
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Fig. 2. Control Architecture. 

A generalized model of human-machine system developed by Riley [43] represents an 
information flow that can be systematically modified according to any rule-base to represent a 
particular level of automation in human-machine interaction. This general model represents the 
most complex level of automation embedded in the most complicated form of human-machine 
interaction. However, this model does not provide any means for implicit communication. We 
altered Riley's model to develop our framework that contains a distinct information channel for 
implicit affective communication. This new framework (as shown in Figure 1) is able to 
accommodate most human-robot cooperative tasks. In order to keep our presentation of the 
control architecture tractable, we focused on a typical exploration setting in this paper where a 
human and a mobile robot (Oracle) worked together in an unknown environment to explore a 
given region. In this case, the Oracle was expected to behave as an intelligent partner to the 
human. This required Oracle to respond appropriately to the emotional states (i.e., anxiety levels, 
in this case) of the human while not undermining the importance of its own safety and work 
performance. The details of this task can be found in the next section. 

4. Experimental Investigation 

A. Experimental Design - Scope and Limitations 

The objective of the experiment was to develop and implement real-time, emotion-sensitive 
human-robot co-ordination that would enable the robot to recognize and respond to the varying 
levels of user anxiety. In this experiment we simulated a scenario where a human and a robot 
needed to work in close coordination on an exploration task. The experiment consisted of the 
following major components: 1) design and development of tasks that could elicit the targeted 
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affective states from the human subjects; 2) system development for real-time physiological data 
acquisition; 3) development and implementation of an affect recognition system based on 
regression tree technique; 4) design and implementation of a robot control architecture that was 
responsive to human anxiety; and 5) design and implementation of a task prioritization 
technique for Oracle that allowed it to modify its behaviors. 

The experiment to demonstrate implicit communication between a human and a robot 
where the robot could change its behavior to address human affective need was performed 
in two stages: first, physiological signals were recorded from the participants while they 
were involved in carefully designed cognitive tasks (described later) that elicited the target 
affective states; and second, streaming the collected physiological data continuously to the 
Oracle as if it were coming in real-time from the operator. This two-stage arrangement was 
done due to the following practical difficulties: 

1. Eliciting high anxiety in a human-robot interaction task is risky and requires fail-safe 
design. One needs to demonstrate the feasibility of affect-sensitive robot behavior in an 
open-loop manner before one can get IRB permission for closed-loop experiments. 

2. Eliciting anxiety, frustration and other affective states through computer tasks is 
less resource consuming. Doing the same with mobile robots would require longer 
hours of training to run the robots. We would also need to provide safe operating 
conditions to avoid accidents, larger work area and more equipment to 
accommodate robot damage than available to us. 

3. Psychologists have already designed cognitive tasks that have been shown to elicit 
anxiety with high probability. However, there is no similar work on robot task 
design that, with a high probability, will generate anxiety. Especially in a 
laboratory environment it is extremely difficult to design such a task because of 
resource limitations. Since our objective was to be able to detect anxiety when it 
occurs, we employ cognitive tasks without compromising our research objectives. 

The two-part experiment described above is expected to serve as a proof-of-concept experiment 
demonstrating the use of physiological feedback to infer the underlying affective states, which is 
then used to adapt robot behavior. However, it is expected that in the future, better availability of 
resources and better task design would enable us to perform field experiments with professional 
robot operators. These experiments would be closed-loop, where both the physiological state of 
the operator working with a robot will be monitored in real-time and the operator responses to 
change in robot behaviors will be evaluated. However, before such steps can be taken, open-loop 
experiments will be useful to verify the proposed concepts. 

Task Design: To obtain physiological data, human subjects were presented with two cognitive 
computer tasks that elicited various affective states. The two tasks consisted of an anagram solving 
task, and a Pong playing task. The anagram solving task has been previously employed to explore 
relationships between both electrodermal and cardiovascular activity with mental anxiety. 
Emotional responses were manipulated in this task by presenting the participant with anagrams 
of varying difficulty levels, as established through pilot work. The Pong task consisted of a series 
of trials each lasting up to four minutes, in which the participant played a variant of the early, 
classic video game "Pong". This game has been also used in the past by researchers to study 
anxiety, performance, and gender differences. Various parameters of the game were manipulated 
to elicit the required affective responses. These included: ball speed and size, paddle speed and 
size, sluggish or over-responsive keyboard and random keyboard response. 
During the tasks, the participants periodically reported their perceived subjective emotional 
states. This information was collected using a battery of five self-report questions rated on a 
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nine-point Likert scale. Self-reports were used as reference points to link the objective 
physiological data to participants' subjective affective state. Each task sequence was 
subdivided into a series of discrete epochs that were bounded by the self-reported affective 
state assessments. These assessments occurred every three minutes for the anagram task and 
every 2-4 minutes for the Pong task. The participants reported their affective state on a scale 
of 0-9 where indicated the lowest level and 9 indicated the maximum level. 
During the experiment, physiological signals (shown in Tablel) were continuously collected 
during each task. A pair of biofeedback sensors was placed on the distal phalanges of the non- 
dominant hand's index and ring fingers to detect the electrodermal activity. Skin temperature was 
monitored from a sensor placed on the distal phalange of the same hand's small finger; and the 
relative pulse volume was monitored by a photoplethysmograph placed in the distal phalange of 
that hand's middle finger. The ECG was monitored using a two-sensor placement on the 
participants' chest. The EMG was monitored using bipolar placements of miniature sensors over 
the left eyebrow (Corrugator Supercilli), the cheek muscle (Zygomaticus Major) and the shoulder 
muscle (Upper Trapezius). Bioimpedance was recorded through spot electrodes placed 
symmetrically on both sides of the neck and thorax region. 

The sensors and data collection system were wearable. The sensors were small, light weight, non- 
invasive, and FDA approved. As a result, the experimental set-up permitted the participants to 
move with minimal restriction. However, they were not completely free to walk around because 
of the needed ethernet communication with the computer. This restriction did not impede task 
performance since the tasks were presented at the computer terminal. Our objective was to 
demonstrate the feasibility of affect detection and recognition using wearable sensors. Once the 
scientific objective is achieved and the advantages demonstrated, miniaturization of the sensors 
and providing wireless connections would be investigated as future work. 

The second part of the experiment consisted of implementing a real-time human-robot interaction 
framework that would enable Oracle to recognize the human's psychological state through 
continuous physiological sensing, and act accordingly to address the psychological needs of the 
human. Oracle-a mobile robot [45] was used in the implementation of the human-robot 
coordination task. It is a popularly used test bed for behavior-based robotics experiments. We 
provided the high-level controller and software needed to command Oracle to do complicated 
tasks. Controlling the Oracle was done by sending low-level commands from a desktop PC using 
a standard RS-232 serial port. There are several options for controlling Oracle including using a 
desktop PC with a tether cable or a radio datalink. In this experiment commands were wirelessly 
sent to Oracle via a radio datalink that allowed control over the drive motors, heading motion, 
gripper, sensors, etc. All the software development was done in Matlab environment. 
Physiological data acquisition: Physiological signals (shown in Tablel) were continuously 
collected during each task in the first experiment. A pair of biofeedback sensors was placed on the 
distal phalanges of the non-dominant hand's index and ring fingers to detect the electrodermal 
activity. Skin temperature was monitored from a sensor placed on the distal phalange of the same 
hand's small finger; and the relative pulse volume was monitored by a photoplethysmograph 
placed in the distal phalange of that hand's middle finger. The ECG was monitored using a two- 
sensor placement on the participants' chest. The EMG was monitored using bipolar placements of 
miniature sensors over the left eyebrow (Corrugator Supercilli), the cheek muscle (Zygomaticus 
Major) and the shoulder muscle (Upper Trapezius). Bioimpedance was recorded through spot 
electrodes placed symmetrically on both sides of the neck and thorax region. The sensors were 
small, light weight, non-invasive, and FDA approved. As a result, the experimental set-up 
permitted the participants to move with minimal restriction. However, they were not completely 
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free to walk around because of the needed ethernet communication with the computer and sensor 
connections to the amplifiers and transducers. This restriction did not impede task performance 
since the tasks were presented at the computer terminal. Our objective was to demonstrate the 
feasibility of affect detection and recognition using wearable sensors. Once the scientific objective 
is achieved and the advantages demonstrated, miniaturization of the sensors and providing 
wireless connections would be investigated as future work. 

Affect recognition system based on regression tree: When the human-robot exploration 
task started, physiological data collected in the first part of the experiment was sent to 
the robot as if coming from the operator in real time. This physiological data was 
processed to extract relevant features from the signals. A regression tree-based affect 
recognizer (created earlier) was utilized to determine operator's level of anxiety based 
on these features. The affect-recognizer accepted as input physiological features and 
produced as output the probable anxiety level of the operator as a scalar in the range 0- 
9 (where indicated almost no anxiety and 9 indicated very high anxiety level). 
Design and implementation of control architecture: Mixed-initiative based control architecture 
was employed to implement affect-sensitive human-robot interaction. The modified control 
architecture is shown in Figure 1. As seen from the figure, in the top-left "robot input" 
quadrant, Oracle received information from both the world and the human through various 
sensors. The world information was obtained through the infrared sensors, touch sensors, light 
sensors etc. Oracle received this information and inferred the world state. The human-related 
information was obtained through biofeedback sensors that provided physiological signals. 
This information was used to interpret the affective (emotional) state of the human, for 
example to judge whether the human is getting stressed, fatigued, or inattentive (implicit 
communication). Oracle also received explicit commands from the human through the control 
sensors (explicit communication). The human's intention was combined with his/her 
emotional state employing context based reasoning to predict the type of the situation. Three 
triggers indicating the type of the situation (Class 1, Class 2 and Class 3) were generated 
depending upon the implicit and explicit communication from the human. The details are 
shown in Figure 4. For instance, if Oracle implicitly sensed that the human was showing a high 
level of anxiety and his/her explicit command was "Come to my assistance immediately", 
Oracle interpreted this as a high priority level situation and classified it as a Class 1 trigger. On 
the other hand, if the anxiety level of the human was low and he/she explicitly commanded 
Oracle to continue its own task of exploration then Oracle interpreted this as a low priority 
situation and assigned Class 3 trigger to it. 
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Fig. 3. Six-Tier Figure 2 Subsumption Model. 



Fig. 4. Generating Triggers from Implicit 
and Explicit Communication. 
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The "robot output" quadrant contained nodes that determine the behavior of Oracle. Oracle 
used its representation of the world, knowledge of its own goals and the urgency level of 
the situation to determine the best course of action. Oracle was assisted by a Six-tier 
subsumption model as shown in Figure 3 to determine the priorities of the tasks. Class 1, 
Class 2 and Class 3 behaviors associated with the respective triggers. More details on these 
behaviors are given in the following paragraph. A behavior on top subsumed or suppressed 
the behavior below it. Hence, Class 1 behavior had the highest priority and exploration had 
the lowest priority. Oracle's decision resulted either in physical motion or initiation of 
speech by Oracle. As seen from the "human input" quadrant, the human received 
information from the world as well as Oracle. The human could perceive the dialogues 
initiated by Oracle and observe its behavior. He/she then exploited this knowledge to infer 
Oracle's state as well as predict what it might do next. Such inference along with the world 
representation that the human formed and his/her own ultimate goals was employed by the 
human to determine the next action. The resulting human's action could be to simply 
monitor Oracle's actions or issue a command to it. Depending on the situation, the human 
could also decide to not do anything. Hence, in each cycle of the loop, there was a 
methodical information flow among the world, Oracle and the human. At the very 
fundamental level, this is a sense-infer-plan-act loop involving both explicit and implicit 
information wherein, Oracle and the human utilized the available information to interact 
with each other and to take actions that influenced each other and the world. Task 
prioritization technique for Oracle: The behavior modifications of Oracle depended upon 
the level of operator anxiety that was detected. Oracle's basic tasks included: 

(i) Exploring the workspace 

(ii) Avoiding obstacles in the workspace 

(iii) Wall following 

(iv) Providing environment related information to the human. 

(v) Responding to the urgency of the situation in the following manner: • 

Class I Trigger -Raise an alarm, Send warning signal to the computer, reach the 

human in shortest possible time 
Class II Trigger - Move to the vicinity of the operator 

Class III Trigger-Initiate a dialogue with the operator to inquire or give suggestions 
The priorities of the execution of the above tasks were decided by a 6-tier subsumption 
model, which has been discussed in detail previously. At any given time Oracle's sensors 
provided it with information regarding the world state and the operator state. The infrared 
range finder and the touch sensors gave information regarding the obstacles in the 
workspace, the compass indicated the orientation of Oracle, the optical encoders indicated 
the motor speed and distance traveled by Oracle and the biofeedback sensors gave an 
indication of the emotional state of the human. 
B. Results 

Results of the two-part experiment described above are given in the following sections. Affect 
Recognition : Fifteen data sets were collected (one for each participant.) Each data set contained six 
hours of physiological signal recording. This was in addition to the ten minutes of baseline (or 
reference) data on each day. Each participant completed approximately 100 task epochs. These 
sessions spanned the anagram and Pong tasks. Wearable sensors were used to continuously 
monitor the person's physiological activities, and the physiological features as mentioned in 
Section III(A) were calculated using customized algorithms. The self-reports of the participants 
indicated the affective state of the person at various times while performing the tasks. 
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There were significant correlations observed between the physiology of the participants and their 
self-reported anxiety levels. This was in accordance with the claim of psychophysiologists that 
there is a distinct relationship between physiology and affective states of a person. It was also 
observed that the physiological activity associated with each affective state was distinct and the 
transition from one affective state to another was accompanied by dynamic shifts in physiological 
activities. Due to the phenomena of person stereotypy no two participants had exactly the same 
set of useful (highly correlated) features. Figure 5 shows the physiological features that were 
highly correlated with the state of anxiety for participant 5 and the corresponding correlation of 
the same features with the state of anxiety for participant 11. An absolute 

correlation greater than equal to 0.3 was considered significant. It can be seen from Figure 5 that 
two features - mean of pulse transit time (PTTmean) and mean of temperature (Temp mean) are 
correlated differently for the two participants. While both were correlated positively with anxiety 
for participant 11, they were negatively correlated for participant 5. However, features like mean 
interbeat interval of impedance (IBI ICGmean), sympathetic activity power (Sym) and mean 
frequency of EMG activity from zygomaticus major (Zfreqmean) were similarly related for both 
participants. However, since our approach to affect detection and recognition was person-specific, 
such expected differences among participants did not create problems. 
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Fig. 5. Person Stereotypy for the affective state of anxiety. 

In spite of differences across participants, there were some features that were more useful 
than the others across all the participants. It was seen that mean of pre-ejection period 
(PEPmean) was a useful feature for nine out of fifteen participants when detecting anxiety. 
Similarly mean interbeat interval derived from ECG (IBI ECGmean) was another important 
feature that was well correlated with anxiety for seven participants. 

It was also observed that each affective state for any participant had a unique set of feature 
correlates- that is the set of features correlated with anxiety were distinct from those correlated 
with boredom or engagement. Since the signature of each affective state was different, it was 
expected that a distinction between anxiety and boredom/ engagement/ anger/ frustration could 
be made based on the physiological features alone. Figure 6 shows the average percentage 
accuracy in distinction between anxiety and other states across the fifteen participants. These 
values were calculated using the method of confusion matrix. It can be seen that on the basis of 
physiology alone, a state of anxiety could be distinguished from a state of anger 82% of the times, 
from state of frustration 76% of the times and from states of engagement and boredom 85% and 
86% of the times respectively. 
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Fig. 6. Regression Tree. 



As mentioned in Section III(B), regression tree methodology was employed to detect affective state 
from physiological features. One of the regression trees generated is shown in Figure 7. As can be 
seen, the attribute that was found most useful for splitting the data was the mean interbeat interval 
of the blink activity (IBI Blinkmean). If the IBI Blinkmean was more than 2.4, the next attribute 
used for splitting was mean amplitude of the PPG peak (PPG Peakmean). If this values was less 
than -0.04, then the predicted anxiety index was six else it was four. Similar splitting was 
performed along the other branches of the regression tree. The numerical values seen are obtained 
after baselining the actual values using the following equation: 



.11".. 



AV K 



eq. 3 



Where AVB is the attribute value after baselining, AVR is the attribute value during the baseline 
period (or the reference value), and AVC is the attribute value during the current period. 
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Fig. 7. Classification accuracy of anxiety with regard to other affective states. 
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Human-Robot Exploration task: The operator and Oracle embarked on an exploration task 
in which Oracle moved around the workspace, avoiding obstacles and following the wall, 
while the operator remained stationed at her desk. Oracle's task was that of exploring the 
workspace and giving relevant feedback regarding the environment to the human from time 
to time. The operator remained at her desk performing her own tasks exclusive of the 
Oracle's behavior. Oracle was sensitive to the anxiety level of the human operator and used 
its own interpretation to determine the nature of the situation. Other factors that Oracle 
considered while planning its next move were the state of the world and its own goals. It 
utilized the six-layer subsumption model described earlier to find out the priority of the 
various tasks that needed to be fulfilled at any given time. 

Since it was not feasible to stress a human subject every time we ran an experiment, the 
physiological signals were recorded during a separate session. These signals were transmitted 
unaltered to Oracle at the time of the experiment in the same manner as if they were being 
recorded in real-time. Ten sessions were conducted during which physiological data from ten of 
the fifteen participants was used. The five participants whose data was not used did not show 
sufficient variation in their anxiety. Figure 8(a) shows the rate of false alarms (when high anxiety 
was detected falsely) and missed alarms (when high anxiety was misclassified as low). The 
average rate of false alarms was 10.55 % while the rate of missed alarm was 12.48 % across the ten 
participants. The average rate of accuracy as seen from Figure 8(b) was 76.97 %. 
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Fig. 8. Behavior Adaptation by oracle. 

Figure 9 demonstrate the capability of Oracle -the emotion-sensitive robot to adapt its behavior 
based on the affective state of the operator. In this figure a 20 minute task session has been 
presented during which the different behaviors of Oracle based on the various triggers are shown. 
Oracle combines the information regarding the human's anxiety level obtained through the 
implicit communication channel with the information regarding the human's intent obtained 
through the explicit communication channel to determine whether it is a Class I, II or III trigger. 
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Figure 4 in Section III shows the matrix that is utilized to interpret the implicit and explicit 
communication in order to evaluate the situation. Figure 9 shows how the behavior of Oracle 
changes according the various triggers that it generates or receives from the operator. It can be 
seen that a Class 3 trigger is generated while Oracle is in the wander behavior (Point A). This 
activates Class 3 behavior in which Oracle suspends its exploration and initiates a speech-based 
dialogue informing the operator that it (Oracle) will be available for help. Oracle then continues its 
exploration task till an information feedback trigger is received from the operator polling Oracle 
for certain task details. Oracle subsumes it wandering behavior and switches to the information 
feedback behavior. It can be observed that Oracle remains in the wandering behavior most of the 
times and switches amidst the other behaviors as and when the relevant triggers are activated. It 
can be seen that in the second half of the task, Class I trigger and survival trigger are received at 
the same time (Point B). Class I trigger having the higher priority induces the Oracle to move from 
wandering behavior to Class I behavior. The Oracle raises an alarm and then processes the 
survival trigger which was previously sent to the cache. 

C. Contributions and Discussion 




Fig. 9. (a) Average rate of false and missed alarms, and (b) Average rate of accuracy in 
anxiety prediction. 

The main contributions of this work as demonstrated by the above experimental results are: 
1) Physiological signals are a powerful indicator of underlying affective states and can play 
an important role in implicit communication between a human and a robot. 2) There exist 
both person stereotypy and context stereotypy. Our method of emotion recognition is both 
person specific and context specific to overcome these problems. And 3) A robot control 
architecture can be designed that can integrate both explicit and implicit communication 
between a human and a robot. In this specific experiment we have demonstrated that a 
robot that works with a person can detect the anxiety of the person and can act on it 
appropriately. These actions will depend on need of the situation and the capability of the 
robot. Anxiety was chosen to be the target affective state in this work because it plays an 
important role in various human-machine interaction tasks that can be related to task 
performance. Detection and recognition of anxiety is expected to improve the 
understanding between humans and robots. However, this framework is not limited to 
detecting anxiety alone and can be used for other affective states as well. Our objective here 
was to demonstrate that such a framework could be realized. It is, to our knowledge, one of 
the first works in the field of human-robot interaction where the robot could be made 
sensitive to human anxiety. Specific set of robot actions will be functions of specific missions 
and are beyond the scope of this work. 
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Related works in the area of affective computing have been summarized in Section I. Most works 
focus on methods for recognizing a person's discrete emotional states while deliberately 
expressing pure emotions such as joy, grief, anger, etc. A mong the other works that detect 
affective states of people engaged in real-life tasks, most use overt signals such as facial 
expressions, voice or speech. Even those that use physiology have not used context-and person- 
specific techniques for learning physiological patterns. Also, there is no known human-robot 
interaction framework functional in this context that uses such a comprehensive set of 
physiological features for real-time emotion detection and robot behavior adaptation. In this work 
we detect the anxiety level purely on the basis of physiological signals. The objective of the 
presented work is to detect and isolate anxiety along a continuous axis. We collected physiological 
data from participants engaged in real cognitive computer-based tasks. The indices derived from 
these biological signals were used for recognition of affective states. 

Currently the presented human-robot interaction based on affective communication 
framework has been verified in open-loop experiments. The rationale for conducting open- 
loop experiments have been explained in Section IV. We are planning to conduct closed-loop 
experiments in the future. However, that will not be possible in our laboratory environment 
because of space and resource limitations. We are currently seeking collaboration for field 
experiments where professional robot operators can participate in the study. 

5. Conclusion and Future Work 

An approach to human-robot interaction that can utilize implicit affective communication along 
with explicit communication is presented. In this work we focus on the state of anxiety as the 
target affective state. A set of physiological indices have been presented that showed good 
correlation with anxiety. The affect recognition technique infers the underlying affective state of 
the human from peripheral physiological signals using regression theoretic methodology. A 
control architecture is presented that can integrate the affective information with the operator's 
explicit information to achieve a versatile and natural human-robot interaction. 
An experiment was conducted where human-robot communication was enhanced by 
enabling the robot to sense human anxiety. In order to perform this experiment two separate 
cognitive tasks were designed to elicit anxiety. 15 human subjects took part in this study 
where each participant involved in the cognitive tasks for 6 hours. Since it was not possible 
to create anxiety while working with the robot within laboratory environment because of 
both resource limitations and safety issues, these data from the cognitive tasks were used in 
real-time for the human-robot experiment. The experiment demonstrated that if such data is 
available, i.e., if a human experiences anxiety while working with a robot, the robot could 
detect and interact with the human in real-time. Thus the experiment demonstrated for the 
first time to our knowledge that human-robot interaction can be performed where affective 
cues can play an important role. Future work will involve in performing closed-loop 
experiments as well as detecting other affective states. 
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1. Introduction 

The chapter describes our current work in developing cognitive robotics architectures in the 
context of robot soccer coaching using spoken language. The work exploits recent developments 
in cognitive science, particularly notions of grammatical constructions as form-meaning mappings 
in language, and notions of shared intentions as distributed plans for interaction and collaboration. 
We exploit social interaction by structuring communication around shared intentions that guide 
the interactions between human and robot. We demonstrate this approach in robot soccer 
coaching distinguishing among three levels of human-robot interaction. The first level is that of 
commanding or directing the behavior of the robot. The second level is that of interrogating or 
requesting a behavior explanation from the robot. The third and most advanced level is that of 
teaching the robot a new form of behavior. The chapter is organized as follows: (i) we explore 
language communication aspects between humans and robots; (ii) we analyze RoboCup soccer, in 
particular the four-legged league, as cognitive platform; (iii) we describe current experiments and 
results in human-robot coaching in the four-legged league; and, (iv) we provide a discussion on 
current and future directions for this work. 

Ideally, research in Human-Robot Interaction will allow natural, ergonomic, and optimal 
communication and cooperation between humans and robotic systems. In order to make progress 
in this direction, we have identified two major requirements: First, we must work in real robotics 
environments in which technologists and researchers have already developed an extensive 
experience and set of needs with respect to HRI. Second, we must develop a domain independent 
language processing system that can be applied to arbitrary domains and that has psychological 
validity based on knowledge from social cognitive science. In response to the first requirement 
regarding the robotic context, we have studied two distinct robotic platforms. The first, the Event 
Perceiver is a system that can perceive human events acted out with objects, and can thus generate 
descriptions of these actions. The second is the Sony AIBO robot having local visual processing 
capabilities in addition to autonomous mobility. In the latter, we explore human-robot interaction 
in the context of four-legged RoboCup soccer league. From the psychologically valid language 
context, we base the interactions on a model of language and meaning correspondence developed 
by Dominey et al. (2003) having described both neurological and behavioral aspects of human 
language, and having been deployed in robotic contexts, and second, on the notion of shared 
intentions or plans by Tomasello (2003) and Tomasello et al. (2006) that will be used to guide the 
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collaborative interaction between human and robot. In section 2 we describe our spoken language 
approach to cognitive robotics; in section 3 we overview the RoboCup four-legged soccer league; 
in section 4 we describe current experimental results with the Sony AIBO platform in human- 
robot interaction; section 5 provides conclusions. 

2. Cognitive Robotics: A Spoken Language Approach 

In Dominey & Boucher (2005a, 2005b) and Dominey & Weitzenfeld (2005) we describe the 
Event Perceiver System that could adaptively acquire a limited grammar based on training 
with human narrated video events. An image processing algorithm extracts the meaning of 
the narrated events translating them into action(agent, object, recipient) descriptors. The event 
extraction algorithm detects physical contacts between objects, see Kotovsky & Baillargeon 
(1998), and then uses the temporal profile of contact sequences in order to categorize the 
events. The visual scene processing system is similar to related event extraction systems that 
rely on the characterization of complex physical events (e.g. give, take, stack) in terms of 
composition of physical primitives such as contact, e.g. Siskind (2001) and Steels & Bailled 
(2002). Together with the event extraction system, a speech to text system was used to 
perform translations sentence to meaning using different languages (Dominey & Inui, 2004). 



2.1 Processing Sentences with Grammatical Constructions 

Each narrated event generates a well formed <sentence, meaning> pair that is used as input to a 
model that learns the sentence-to-meaning mappings as a form of template in which nouns and 
verbs can be replaced by new arguments in order to generate the corresponding new meanings. 
These templates or grammatical constructions, see Goldberg (1995) are identified by the 
configuration of grammatical markers or function words within the sentences (Bates et al., 1982). 
Each grammatical construction corresponds to a mapping from sentence to meaning. This 
information is also used to perform the inverse transformation from meaning to sentence. 
For the initial sentence generation studies we concentrated on the 5 grammatical 
constructions shown in Table 1. These correspond to constructions with one verb and two or 
three arguments in which each of the different arguments can take the focus position at the 
head of the sentence. On the left example sentences are presented, and on the right, the 
corresponding generic construction is shown. In the representation of the construction, the 
element that will be at the pragmatic focus is underlined. 





Sentence 


Construction < sentence, meaning> 


1 


The robot kicked the ball 


< Agent event object, event(a%ent, object> 


2 


The ball was kicked by the robot 


<Object was event by agent, event(agent, object> 


3 


The red robot gave the ball to the 
blue robot 


< Agent event object to recipient, 

event(agent f object, recipient)> 


4 


The ball was given to the blue 
robot by the red robot 


<Object was event to recipient by agent, 
event(agent f object recipient)> 


5 


The blue robot was given the ball 
by the red robot 


<Recipient was event object by agent, 
event (agent object recipients 



Table 1. Sentences and corresponding constructions. 

This construction set provides sufficient linguistic flexibility, for example, when the system is 
interrogated about the red robot, the blue robot or the ball. After describing the event give(red robot, 
blue robot, ball), the system can respond appropriately with sentences of type 3, 4 or 5, respectively. 
Note that sentences 1-5 are specific sentences that exemplify the 5 constructions in question, and 
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that these constructions each generalize to an open set of corresponding sentences. 
We have used the CSLU Speech Tools Rapid application Development (RAD) (CSLU, 2006) 
to integrate these pieces, including (a) scene processing for event recognition, (b) sentence 
generation from scene description and response to questions, (c) speech recognition for 
posing questions, and (d) speech synthesis for responding. 

2.2 Shared Intentions for Learning 

Perhaps the most interesting aspect of the three part "command, interrogate, teach" scenario 
involves learning. Our goal is to provide a generalized platform independent learning 
capability that acquires new <percept, response> constructions. That is, we will use existing 
perceptual capabilities, and existing behavioral capabilities of the given system in order to 
bind these together into new, learned <percept, response> behaviors. 

The idea is to create new <percept, response> pairs that can be permanently archived and used in 
future interactions. Ad-hoc analysis of human-human interaction during teaching-learning reveals 
the existence of a general intentional plan that is shared between teachers and learners, which 
consists of three components. The first component involves specifying the percept that will be 
involved in the <percept, response> construction. This percept can be either a verbal command, or 
an internal state of the system that can originate from vision or from another sensor. The second 
component involves specifying what should be done in response to this percept. Again, the 
response can be either a verbal response or a motor response from the existing behavioral 
repertoire. The third component involves the binding together of the <percept, response> 
construction, and validation that it was learned correctly. This requires the storage of this new 
construction in a construction database so that it can be accessed in the future. This will permit an 
open-ended capability for a variety of new types of communicative behavior. 
In the following section this capability is used to teach a robot to respond with physical 
actions or other behavioral responses to perceived objects or changes in internal states. The 
user enters into a dialog context, and tells the robot that we are going to learn a new 
behavior. The robot asks what is the perceptual trigger of the behavior and the human responds. 
The robot then asks what is the response behavior, and the human responds again. The robot 
links the <percept, response> pair together so that it can be used in the future. 
Having human users control and interrogate robots using spoken language results in the 
ability to ergonomically teach robots. Additionally, it is also useful to execute components 
of these action sequences conditional on perceptual values. For example the user might 
want to tell the robot to walk forward until it comes close to an obstacle, using a "command 
X until Y" construction, where X corresponds to a continuous action (e.g. walk, turn left) 
and Y corresponds to a perceptual condition (e.g. collision detected, ball seen, etc.). 

3. RoboCup Soccer: Four-Legged League 

In order to demonstrate the generalization of the spoken language human-robot interaction 
approach we have begun a series of experiments in the domain of RoboCup Soccer (Kitano, 1995), a 
well documented and standardized robot environment thus provides a quantitative domain for 
evaluation of success. For this project we have chosen as testing platform the Four-Legged league 
where ITAM's Eagle Knights team regularly competes (Martinez-Gomez et al. 2005; Martinez- 
Gomez & Weitzenfeld, 2005). In this league two teams of four robots play soccer on a 6m by 4m 
carpeted soccer field using Sony's Four-Legged AIBO robots (RoboCup, 2004), as shown in Figure 
1. Robots in this league have fully autonomous processing capabilities including a local color-based 
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camera, motors to control leg and head joints, and a removable memory stick where programs can 
be loaded. The robots also include wireless communication capabilities to interact with other robots 
in the field as well as computers outside. In addition to the two colored goals, four colored cylinders 
are used in helping the robots localize in the field. To win, robots need to score as many goals as 
possible in the opposite goal. Ball is orange and robots use either a red or blue colored uniform. As 
in human soccer, good teams need to perform better that opponents in order to win, this includes 
being able to walk faster, process images, localize and kick the ball in a more efficient way, and have 
more advanced individual behaviors and more evolved team strategies. 




Fig. 1. Four-Legged Soccer League. Two teams of four robots play against each other in a 6m by 
4m carpet. AIBOs from Sony are used having fully autonomous control. Sensors include a local 
color-based camera having images processed by a local CPU sending output commands to 
motors controlling joints in the four legs and heads. The AIBO also includes wireless 
communication capabilities to interact with other robots in the field and computers outside. 
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Fig. 2. Four-Legged System Architecture. The system includes the following processing 
components: sensors, actuators, motion, vision, localization, behaviors and wireless 
communication. The latter is used to share information with other robots through the game 
controller responsible of informing all robots of the state of the game. 
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A typical four-legged system architecture shown in Figure 2 consists of the following modules: 

• Sensors. Primary sensors include a color camera and feedback from motors. The raw 
camera image is passed to a vision module segmenting objects of interest. Other 
sensors are used to obtain complementary information on the robot such as joints. 

• Actuators. The main robot actuators are head and leg motor controlling joints for 
walking and head turns. 

• Motion. The motion module is responsible for robot movements, such as walk, run, 
kick, pass, turn, move the head, etc. It receives commands from the behavior module 
with output sent to the corresponding actuators representing individual leg and head 
joint motor control. Robot motions are adapted according to team roles, for example, 
the goalie has different defensive poses in contrast to other team players. This also 
applies to different head and ball kicks. 

• Vision. The vision module receives a raw image from the camera segmenting objects 
according to color and shape. Objects recognized include ball, robots, cylinder 
landmarks and goals. More details are given in Section 3.1. 

• Localization. The localization module uses visual information to provide a reliable 
localization of the robot in the field. Colored cylinders, goals and white lines are used 
for this task. More details are given in Section 3.2. 

• Behaviors. The behaviors module makes decisions affecting individual robots and team 
strategies. It takes input from sensors and localization system to generate commands sent 
to motion and actuators modules. Further details are given in Section 3.3. 

• Wireless Communication. Robots include wireless communication to share 
information and commands with the external Game Controller or among robots. 
Data transmitted includes information such as player id, location of ball if seen, 
distance to the ball, robot position and ball position. 

3.1 Vision 

The vision module segments incoming camera images to recognize objects of interest from 
color and shape information. The vision architecture consists of the following stages: image 
capture, color calibration, segmentation, recognition and identification, as shown in Figure 
3. The vision architecture is common to many leagues in RoboCup, including the Mid-size 
league havign a global viewing camera and the Small-size league having an aereal camera 
(Martinez-Gomez & Weitzenfeld, 2004). In order to recognize and identify objects of interest in 
the image appropriate calibration needs to be performed to adapt to existing lighting conditions. 
We take initial photos of objects of interest and then select colors that we want to distinguish 
during segmentation. Figure 4 shows sample output of the segmentation calibration process. 
Images on the first and third columns are segmented to images in the second and fourth 
columns, respectively. Colors of interest are orange, green, yellow, pink and blue. All other colors 
are left as black in the images. 
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Fig. 3. Vision Architecture. The vision module includes the following stages: image capture, 
color calibration, segmentation, recognition and identification. Output is in the form of 
recognized and identified objects. 
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Fig. 4. Color segmentation. Images on the first and third columns are segmented to images 
in the second and fourth columns, respectively. Colors of interest are orange, green, yellow, 
pink and blue. All other colors are left as black in these images. 

After color regions are obtained, objects are recognized according to certain requirements to allow 
some confidence that the region being analyzed corresponds to an object of interest. For example, 
the ball must have green in some adjacent area with a similar criteria used to identify goals. The 
recognition of landmarks is a little more complex, nevertheless after more elaborated comparison 
landmarks are identified. Finally, recognized objects are identified depending on color 
combinations, for example, preestablished landmark "1" and landmark "T ' . 



3.2 Localization 

To be successful robots need to localize in the field in an efficient and reliable way. 
Localization includes computing distances to known objects or landmarks, use of a 
triangulation algorithm to compute exact positioning, calculate robot orientation angles, and 
correct any resulting positioning errors, as described in Figure 5. 

Distances to Objects. The first step in localizing is to obtain distances from the robot to identified 
objects in the field. The more objects the robot can distinguish in the field the more reliable the 
computation. After making several experiments, we developed a simple algorithm that computes 
distances to objects by using a cubic mathematical relationship that takes as parameter the viewed 
object area and returns the distance to that object. The distance range tested was from 15 
centimeters to 4 meters. Closer than 15 cm or beyond 4 m it becomes harder to compute distances 
or distinguish between objects and noise, respectively. 
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Fig. 5. Localization system block diagram. Localization involves computing distances to 
known objects in the field, triangulations based on landmarks and goals, angles to 
landmarks and goals, and computing error corrections to obtain reliable positioning. 
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Triangulation Algorithm. Following distance computation we apply a triangulation method 
from two marks to obtain the position of the robot in the field. Triangulation results in a very 
precise positioning of the robot in a two dimensions plane. If a robot sees one landmark and can 
calculate the distance to this landmark, the robot could be anywhere in a circumference with 
origin in the landmark, and radio equal to the distance calculated. By recognizing two landmarks 
the robot can compute its location from the intersection of two circumferences, as shown in 
Figure 6. Note that the robot could be in one of two intersection points in the circumferences, 
although one of these two points will fall outside the field of play. 




Fig. 6. Triangulation from two landmarks. By calculating distance to two different 
landmarks the robot can compute its position in the field. 

Angle Calculation. Once robot position is computed orientation is calculated to complete 
localization. Two vectors are calculated with origin at the robot pointing to the marks used 
as references for triangulation, as shown in Figure 7. Orientation calculation is usually more 
precise than positioning while also being more important to the game. Kicking the ball in 
the right direction is quite critical to winning. 




Fig. 7. Robot orientation. By computing vectors to marks (or goals) the robot may calculate 
its orientation in the field. 

Correction Algorithm. Robot positioning resulting from the localization algorithm usually results 
in inconsistencies between contiguous frames. To stabilize localization computation correction 
algorithms are necessary starting by smoothing historic measurements. To reduce variation of the 
output signal for the triangulation algorithm the following filter function shown in equation 1 can 
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be used where s(x) represents the updated position value as a function of previously computed x 
position values taking an average over n historic samples. 



I>) 



s(x) = 



(1) 



Figure 8 shows sample output from this filter correction. Original signals can produce 
variations of 10% in contiguous positions. By applying the filter this variation can be 
reduced to less than 3%, see Martinez-Gomez & Weitzenfeld (2005) for more details. 
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Fig. 8. Correction Algorithm. Abrupt variations in positioning are caused by resolution related 
errors. To reduce this variability data can be smoothed by averaging with a historic filter. 

In addition to variances in positioning, the large size of the robots, around 20cm in length and 
10cm in width, requires a positioning precision of at least half the size of the robot. Furthermore, 
positioning in the field like in human soccer does not require exact knowledge of location as 
opposed to orientation. For this purpose localization by field regions can be more effective than 
knowing exact positioning. In Figure 9 the complete field is divided into twelve similarly sized 
areas to provide rough localization in the field. During experimentation, localization in some 
regions resulted in larger errors due to changes in illumination. 
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Fig. 9. Localization by regions. While orientation in the field is critical in moving and kicking 
the ball in the right direction, positioning does not require very high precision as in human 
soccer. Knowing positioning in relation to certain regions in the field provides enough 
information for play as shown in the diagram with a 3 by 4 field subdivision. 
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Regions can be defined in a heterogeneous way as well, e.g. having a specific local goal area 
while dividing middle field areas with coarser granularity than areas closer to the goals. 
Also, probabilistic methods not described in the chapter are usually used to localize when 
occlusions occur during a game. 



3.3 Behaviors 

The behavior module receives input information from sensors, vision and localization in 
order to compute individual and team behavior. Output from behavior decisions are sent to 
motion and actuators. In defining team robot behaviors, we specify different player roles, 
e.g. Goalkeeper, Attacker, and Defender. Each role behavior depends on ball position, state 
of game and overall team strategy. 
Goalkeeper basic behaviors are described by a state machine as shown in Figure 10: 

• Initial Position. Initial posture that the robot takes when it is turned on. Depending 
on its ability to localize robot may autonomously move to its initial position. 

• Search Ball. An important aspect of the game is searching for the ball around the 
field. If communication is enabled among robots, searching may be made more 
efficient by doing this task as a team with individual robots informing others where 
the ball is when found. 

• Search Goal. The ultimate objective of the goalkeeper is to defend its own goal. To 
achieve this it must always know its relative position. Sometimes the goalkeeper gets 
away from its area for a special defensive move and needs to return to the goal by 
searching around. 

• Reach Ball. After searching and finding the ball, the robot can walk towards it in 
order to take possession or simply kick the ball. Additional plays include reaching 
the ball up to certain distance in order to defend the goal from a possible ball kick by 
an opponent. 

• Reach Goal. After searching and finding the goal, the goalkeeper moves towards it 
to relocate on the goal line in the middle of the goal. 

• Kick ball. The goalkeeper can kick the ball in different ways to get it out of its own goal. 



csft'ise&beij 


Gan-WCOAirollOf Plying 


( Initial Position j 


itebefl 


Search Goal } 

Reach goal 






Cantsaebal 


M, Search ball I 


Pcm Septal X 

\ Reach ball \ 


SeebsllV 
s^ — — --QartJ 

( Kick Ball j 


OisiMC'Wtiaii^Ki-qn 





Fig. 10. Goalkeeper Basic Behavior. The basic individual goalkeeper state machine includes 
activities starting from an initial position followed by search ball, search goal, reach ball, 
reach goal and kick ball. 
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Attacker and Defender basic common individual behaviors are described by a state 
machine as shown in Figure 11: 

• Initial Position. Initial posture that the robot takes when it is turned on. Depending 
on the ability to localize robot may autonomously move to their initial positions. 

• Search Ball. An important aspect of the game is searching for the ball around the field. If 
communication is enabled among robots, searching may be made more efficient by doing 
this task as a team with individual robots informing others where the ball is when found. 

• Reach Ball. After searching and finding the ball, the robot can walk towards it in 
order to move with it or kick the ball. Additional plays include turning with the ball 
and reaching the ball with different orientations. 

• Explore Field. Exploring the field is a more extended search than the search ball 
behavior in that it can walk throughout the arena not only looking for the ball but 
also searching for goals and landmarks. 

• Kick Ball. The robot can kick the ball in different ways such as to score a goal in the 
case of an attacker or to simply kick it forwards in the case of a defender. 
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Fig. 11. Attacker and Defender Basic Behavior. The basic individual attacker and defender 
state machine includes common activities starting from an initial position followed by 
search ball, explore field, reach ball, and kick ball. 

4. Human-Robot Coaching in RoboCup Soccer 

While no human intervention is allowed during a RoboCup Four-Legged soccer league 
game, in the future humans could play a decisive role analogous to real soccer coaches. 
Coaches could be able to adjust in real-time their team playing strategies according to the 
state of the game. RoboCup already incorporates a simulated coaching league where 
coaching agents can learn during a game and then advice virtual soccer agents on how to 
optimize their behavior accordingly, see (Riley et al., 2002; Kaminka et al. 2002). In this 
section we describe our most recent work in human-robot interaction with Sony AIBOs. 



4.1 Human-Robot Architecture 

The human-robot interaction architecture is illustrated in Figure 12. The spoken language 
interface is provided by the CSLU-RAD framework while communication to the Sony AIBO 
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robots is done in a wireless fashion via the URBI platform (URBI, 2006). The URBI system 
provides a high level interface for remotely controlling the AIBO. Via this interface, the 
AIBO can be commanded to perform different actions as well as be interrogated with 
respect to various internal state variables. Additionally, URBI provides a vision and motion 
library where higher level perceptions and movements can be specified. (The AIBO 
architecture shown at the right hand side of Figure 12 describes the robot processing 
modules previously shown in Figure 2.) 
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Fig. 12. CSLU-URBI-AIBO system architecture. The left portion of the diagram shows a 
Human Coach interacting with the CSLU RAD spoken language system that in turns 
interacts via wireless communication with the URBI interface at the AIBO. The diagram to 
the right shows the internal AIBO processing modules: Sensors, Actuators, Vision, Motion, 
Localization, Behaviors and Wireless Communication. 



4.2 Command, Interrogate and Teach Dialogs 

In order to demonstrate the human coaching model we have developed and experimented 
with simple dialogs that let the user: (1) command the robot to perform certain actions 
including perception related actions; (2) interrogate the robot with specific questions about its 
state and corresponding perceptions; and (3) teach the robot to link a sequence of lower level 
behaviors into higher level ones. 

Command. We define a set of action-only and action perception commands. Action-only 
commands i.e. no perception, include: Stop, Move, Turn, Turn Head, and Kick Ball Depending 
on the commands, these may include arguments such as magnitude of rotation, and 
movement in degrees or steps, etc. For example a rotation command would be Turn 180 
degrees and a movement command would be Move 4 steps. It should be noted that at this 
level commands such as Kick Ball would not use any perceptual information, i.e. the 
resulting kick will depend on the current robot orientation. We also define a set of action- 
perception commands requiring the full perception-action cycle, i.e. the action to be 
performed depends on the current robot perceptions. These commands include: Kick Ball 
with a specified direction; Reach Ball moving to a position behind the ball pointing towards 
the goal; Initial Position during game initialization requiring localization in the field; Pass the 
Ball to gently kick the ball to another team robot; Move to Location specifying a position in the 
field where to move; Search Ball resulting in robot looking for a ball nearby; Explore Field 
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resulting in a more extensive search for the ball; Defend Goal resulting in all robots moving 
close to the goal requiring knowledge of the robot location in the field; Defend Kick in trying 
to block a kick from the other team, requiring knowledge of ball location, and Attack Goal 
similar although opposite in behavior to defending goal. 

Interrogation. We define state and perception interrogation commands returning 
information on current actions or behaviors. State interrogations include for example: What 
was your last action, e.g. kicked the ball; Why did you take the last action, e.g., I saw the ball, so I 
moved towards it; What is your current behavior, e.g. I'm searching for the ball; What is your 
current role in the game, e.g. I am the goalie. Perception interrogations include for example: 
Do you see the ball returning e.g. I do, I don't; What is your distance to the ball, returning e.g. 30 
centimeters; What is your current orientation, returning e.g. 45 degrees (in relation to field 
coordinate system); What is your current position, returning e.g. I am in region 9; What is the 
position of object X returning an estimate of its position. 

We have already implemented in the AIBO a number of these commands and 
interrogations. A sample command and interrogation dialog is shown in Table 2 
corresponding to "Go get the ball and walk it into the goal" . 



Robot: Welcome to the LAFMI Human-Robot Interaction Demonstration. I am 

the AIBO... 
Human: Do you see the ball ? 
Robot: Looking for the ball. . .1 don't see anything. 
Human: Turn right. 
Robot: Turning right. (Turns right) 
Human: Do you see the ball ? 
Robot: Looking for the ball. .. I see the ball. 
Human: Get the ball. 

Robot: Going for the ball. (Starts walking to the ball). 
Human: Stop. Turn right. 

Robot: Turning right. (Turns right but misses the ball) 
Human: Back up. 
Robot: Back up. (Backs up) 
Human: Turn left. 

Robot: Turning left. (Turns left but misses the ball) 
Human: Turn right. 

Robot: Turning right. (Turns right and catches the ball) 
Human: Walk. 

Robot: Walking. (Starts walking, pushing the ball towards the goal) 
Human: Walk. 
Robot: Walking. (Keeps walking, pushing the ball and "scoring") 



Table 2. Sample human-robot dialog. The dialog shows sample spoken interaction with the 
human for commanding and interrogating the robot. 

Teach. The ultimate goal in human-robot coaching in the context of soccer is being able to 
positively affect the team performance during a game. While part of this interaction can 
eventually be carried out by agent coaches inside the robot, it is our goal to define the basic 
capabilities and communication interactions that human coaches should have. For example, 
being able to transmit strategy knowledge in the form "if blocked pass the ball to player behind". 
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Such a command will modify an internal robot database with "if possess (ball) and goal(blocked) 
then pass (ball)" '. 

Previous systems allowed the user to use spoken language to teach the AIBO robot the 
association between a name and a single behavior in the robot's repertoire (Dominey et al. 
2005). More recently, we have extended this so that the system can associate a sequence of 
commands with a new name in a macro-like capability. The limitations of this approach 
result from the fact that all of the motor events in the sequence are self contained events 
whose terminations are not directly linked to perceptual states of the system. We can thus 
teach the robot to walk to the ball and stop, but if we then test the system with different 
initial conditions the system will mechanically reproduced the exact motor sequence, and 
thus fail to generalize to the new conditions. 

Nicolescu & Mataric (2001, 2003) developed a method for accommodating these problems 
with a formalized representation of the relations between pre-conditions and post- 
conditions of different behaviors. In this manner, after the robot has performed a human 
guided action, such as following the human through an obstacle course and then picking up 
an object, the system will represent the time ordered list of intervals during which each of 
the component behaviors is active. From this list, the pre- and post-condition relations 
between the successive behaviors can be extracted, generalized over multiple training trials, 
and finally used by the robot to autonomously execute the acquired behavior. 
Boucher and Dominey (2006) builds upon these approaches in several important ways. First 
they enrich the set of sensory and motor primitives that are available to be used in defining 
new behaviors. Second, they enrich the human-robot interaction domain via spoken 
language and thus allow for guiding the training demonstrations with spoken language 
commands, as well as naming multiple newly acquired behaviors in an ever increasing 
repertoire. Third, they ensure real-time processing for both the parsing of the continuous 
valued sensor readings into discrete parameterized form, as well as the generalization of the 
most recent history record with the previously generalized sequence. This ensures that the 
demonstration, test, correction cycle takes places in a smooth manner with no off-line 
processing required. 

Here is a simple example scenario with the AIBO. In this case the user will teach the robot a 
form of collision avoidance through demonstration. The user initiates the learning by 
commanding the robot with a spoken command "turn around " that does not correspond to 
a primitive command nor to a previously learned command. The robot thus has no 
knowledge of what to do, and awaits further instructions. The user commands the robot to 
"march forward" and the robot starts walking. The user sees that the robot is approaching a 
wall, and tells the robot to stop. He then tells the robot to turn right. Behind and to the 
right of the robot is the red ball. When the robot has turned away from the wall and is 
facing the ball the user tells it to stop turning, and then tells it that the learned behavior 
demonstration is over. 

Now let us consider the demonstration in terms of the commands that were issued by the user, 
and executed by the robot, and the preconditions that could subsequently be used to trigger 
these commands. The robot was commanded to "turn around." Because it had no representation 
for this action, it awaited further commands. The robot was then commanded to walk. Before it 
collided with the wall the robot was commanded to stop walking. It was then commanded to 
turn right, and to stop when it was in front of the red ball. Now consider the perceptual 
conditions that preceded each of these commands, which could be used in a future automatic 
execution phase to sequentially trigger the successive commands. The pertinent precondition to 
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start walking was that the command to "turn arouncT had been issued. The pertinent 
precondition to stop walking was the detection of an obstacle in the "near" range by the distance 
sensor in the robots face. The pertinent preconditions for subsequently turning right are that the 
robot is near something, and that it has stopped walking. 

The goal then is for the system to encode the temporal sequence of all relations (which 
include user commands and perception values) in a demonstration run, and then to 
determine what are the pertinent preconditions for each commanded action relation. 
Likewise, it may be the case that perceptual relations were observed during the 
demonstration that were not pertinent to the behavior that the human intended to teach the 
robot. The system must thus also be able to identify such "distractor" perceptions that 
occurred in a demonstration, and to eliminate these relations from the generalized 
representation of the behavioral sequence. 

5. Conclusions 

The stated objective of the current research is to develop a generalized approach for human- 
machine interaction via spoken language that exploits recent developments in cognitive science - 
particularly notions of grammatical constructions as form-meaning mappings in language, and 
notions of shared intentions as distributed plans for interaction and collaboration. In order to do 
this, we tested human-robot interaction initially with the Event Perceiver system and later on 
with the Sony AIBOs under soccer related behaviors. We have presented the system architecture 
for the Eagle Knights Four-Legged team as a testbed for this work. 

With respect to social cognition, shared intentions represent distributed plans in which two 
or more collaborators have a common representation of an action plan in which each plays 
specific roles with specific responsibilities with the aim of achieving some common goal. In 
the current study, the common goals were well defined in advance (e.g. teaching the robots 
new relations or new behaviors), and so the shared intentions could be built into the dialog 
management system. We plan to continue this work by experimenting with more evolved 
behaviors in testing full coaching capabilities in the soccer scenario. Videos for several 
human-robot dialogs, including the previous one, can be found in Dominey & Weitzenfeld 
(2006). 
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1. Introduction 



Early communication between a child and a caregiver is mainly embodied through touch and eye- 
contact, which convey various kinds of emotional information (Kaye, 1982; Trevarthen, 2001). This 
communication of emotion develops into joint attention (Butterworth & Jarrett, 1991), where both 
alternate between looking at the same object or event and looking at each other. By mutually 
monitoring emotions and attention in this way, the child and the caregiver share awareness of a 
topical target as well as emotional attitudes towards it. Thus, the child can learn the meaning and 
value of various objects and events in the world, which leads him or her to the acquisition of 
language and culture (Tomasello, 1993, 1999). 

With inspiration from the psychological study of social development, we have developed a child- 
like robot, Infanoid (Kozima, 2002), and a creature-like robot, Keepon (Kozima et al., 2004), as 
research platforms for testing and elaborating on psychological models of human social 
intelligence and its development in real-world settings. We are currently implementing on these 
robots software modules required for embodied interaction with people, especially with children. 
In addition, we are observing and analyzing social development in children when they interact 
with these robots. These two complementary research activities will help us to model social 
communication and its development during the first years of life. 

This paper describes design principles of interactive robots for the cognitive study of human 
social intelligence and for the development of pedagogical and therapeutic services for 
children's social development. After reviewing recent psychological findings on children's 
social development and recent advances in robotics facilitating social interaction with 
children, we discuss design principles that make robots capable of embodied interaction 
with children. We introduce our robotic platforms, Infanoid and Keepon, as examples of 
implementation of these design principles. We then describe how typically-developing 
children interact with Infanoid and Keepon, from which we model how social interaction 
dynamically unfolds as time passes and how such interactions qualitatively change with age. 
We have conducted longitudinal field observations of a group of children with 
developmental disorders and a group of typically-developing preschool children interacting 
with Keepon. We learned from these observations that an appropriately designed robot 
could facilitate not only dyadic interaction between a child and the robot, but also triadic 
interaction among children and carers, where the robot functions as a pivot of the 
interpersonal interactions. Finally, we discuss the possible use of interactive robots in 
pedagogical and therapeutic services for typically-developing children and for those with 
developmental disorders, especially autistic spectrum disorders. 
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2. Background 

2.1 Social Development in Childhood 

Children, especially those in the first year of life, develop the capability for social communication 
through physical and social interactions with their caregivers (e.g., mothers) and artefacts such as 
toys (Kaye, 1982; Trevarthen, 2001). Even neonates have various innate competencies to respond 
to and act on the environment, such as those for detecting and tracking human faces (Fantz, 1961; 
Morton & Johnson, 1991), mimicking orofacial actions, i.e., neonatal imitation (Meltzoff & Moore, 
1977; Nadel & Butterworth, 1999), and recognizing the prosodic features of their mothers' voices 
(DeCasper & Fifer, 1980; Fernald, 1991). These competencies are, of course, the driving force for 
them to interact with the environment; however, to be socially meaningful, a child's acts have to 
be responded to, guided, and given social functions by adults (especially, caregivers). 
Let us briefly look at how social development is initiated and maintained in a supportive 
environment during the first year of life. 

• Under three months of age: 

The child establishes eye-contact with the caregiver along with exchanges of voice and/ or 
facial expressions in the form of rhythmic turn-taking. The temporal structure mainly 
originates from the caregiver's reading of the child's response pattern. (Fig.l, left.) 

• Three to nine months of age: 

The caregiver interprets and actively reponds to the child's mental states, such as 
desire and pleasure or displeasure. Although it is still asymmetric, their interaction 
seems socially meaningful. The child gradually learns to predict the caregiver's 
behavior, which makes the interaction more symmetric. (Fig.l, middle.) 

• Over nine months of age: 

Joint attention, i.e. an activity in which two people look at the same target 

(Butterworth & Jarrett, 1991), emerges in the child-caregiver interactions (Tomasello, 

1999; Trevarthen, 2001). With the help of gaze and/ or pointing, they share perception 

of the target and refer to each other's actions directed at the target (including 

vocalization and facial expressions), thus sharing the emotional meaning or value of 

the target (Dautenhahn, 1997; Zlatev, 2001). (Fig.l, right.) 

The development of child-caregiver interaction during the first year of life establishes the 

basis of an empathetic understanding of each other's mental states (Trevarthen, 2001). With 

this foundation, the child starts learning various social skills like language use, tool use, and 

cultural conventions (Tomasello, 1999; Zlatev, 2001; Kozima & Ito, 2003). 






Fig. 1. Three stages of social development in the first year of life: eye-contact and exchange 
of emotions (left), proto-social interaction by the caregiver's interpretation (middle), and 
joint attention and sharing actions regarding the target (right). 

2.2 Eye-contact and Joint Attention 

Eye-contact and joint attention are fundamental activities that maintain child-caregiver 
interactions. A child and a caregiver spatially and temporally correlate their attention and 
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emotions, in which they refer to each other's subjective feelings of pleasure, surprise, and 
frustration (Dautenhahn, 1997; Zlatev 2001). We believe all communication emerges from 
this mutual reference. 

Eye-contact is the joint action of two individuals looking into each other's face, especially the eyes 
(Fig. 2, left). It serves not only to monitor each other's gaze and facial expressions, but also to 
synchronize interactions and to establish mutual acknowledgment (Kozima et al., 2004), such as 
"My partner is aware of me" and "My partner is aware that I am aware of her." 
Joint attention is the joint action of two individuals looking at the same target by means of 
gaze and/ or pointing (Butterworth & Jarrett, 1991) (Fig. 2, right). First, the caregiver actively 
follows and guides the child's attention so that the child can easily capture the target; then 
the child gradually becomes able to follow and guide the partner's attention. Joint attention 
not only provides the interactants with shared perceptual information, but also with a 
spatial focus for their interaction, thus creating mutual acknowledgment (Kozima et al., 
2004), such as "I and my partner are aware of the same target" and "Both of our actions 
(such as vocalization and facial expressions) are about the target." 





Fig. 2. Exchange of emotions and attention through eye-contact and joint attention: exchange 
of emotions through eye-contact (left), and empathetic understanding through joint 
attention (right). 

2.3 Interactive Robots for Children 

There has been a growing amount of interest in designing interactive robots that can engage 
in social interaction with children. This is motivated not only by pedagogical, therapeutic, 
and entertaining applications of interactive robots, but also by the assumption that the 
underlying mechanism for children's embodied interaction and its development is the 
fundamental substratum for human social interaction in general. 

Motivated by this assumption, a number of research projects in the field of embodied 
interaction have developed interactive robots explicitly for interaction with children. For 
example, Kismet (Breazeal & Scassellati, 2000) is one of the pioneering examples of "sociable 
robots"; Kismet emphasized the elicitation of caretaking behaviour from adults, facilitating the 
robot's learning to communicate with people, but the robot was also effective at facilitating peer 
interaction with children. Another pioneer is the AuRoRa project (Dautenhahn, 1999), which 
reported that even simple mobile robots gave autistic children a relatively repetitive and 
predictable environment that encouraged spontaneous interactions, such as chasing games, 
with the robots. Billard developed a doll-like robot, Robota (Billard, 2002), for mutual imitation 
play with autistic children; Robins intensively analyzed two children playing together with 
Robota and observed mutual monitoring and cooperative behavior to derive desirable 
responses from it (Robins et al., 2004). Scassellati is building and using social robots (Scassellati, 
2005) for the study of social development, especially that of children with autistic spectrum 
disorders. Michaud devised a number of mobile and interactive robots, including Roball and 
Tito, and observed interaction with autistic children in order to explore the design space of 
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child-robot interactions that fosters children's self-esteem (Michaud & Theberge-Turmel, 2002). 
Goan used a creature-like robot, Muu (developed by Okada) to observe child-robot interactions 
mediated by the shared activity of arranging building blocks (Goan et al., 2005). 

2.4 Autistic Spectrum Disorders 

It is notable that several of the studies previously discussed have dealt with autism, or 
autistic spectrum disorders, which is a neurophysiological disorder caused by a specific and 
mainly hereditary brain dysfunction (Frith, 1989). People with autism generally have the 
following major difficulties. 

• Social (non-verbal) interaction: 

They have difficulty in understanding others' intentions and emotions from gaze, 
facial expressions and gestures, and in sharing interests and activities with others. 

• Linguistic (verbal) interaction: 

They have difficulty in verbal communications, especially in pragmatic use of 
language. They also have delayed language development or a lack of it, and 
stereotyped or repetitive speech. 

• Imagination: 

They have difficulty in maintaining the diversity of behaviour and have stereotyped and 
restricted interests and actions. They also have difficulty in coping with novel situations. 
These difficulties limit the ability of autistic people to establish and maintain social relationships 
with others. Researchers in social robotics therefore have a particular interest in autism to better 
understand the underlying mechanisms responsible for social interaction and its development. 

3. Robotic Platforms 

We are presently developing interactive robots for modeling the development of embodied 
social interaction and for investigating the cognitive mechanisms of human social 
development. We describe two robots we have built: Infanoid, an upper-torso child-like 
humanoid, and Keepon, a simple creature-like robot. 

3.1 Infanoid: A Child-like Humanoid 

Infanoid (Fig. 3, right), our primary research platform, is an upper-torso humanoid robot 
that is 480 mm tall, the approximate size of a 4-year-old human child. The latest version of 
Infanoid has 29 actuators (mostly DC motors with digital encoders and torque-sensing 
devices) and a number of sensors arranged in its relatively small body. It has two hands, 
each of which has four fingers and a thumb, capable of grasping small objects, pointing, and 
making a variety of other hand gestures. 

The head of Infanoid has two eyes, each of which contains two different colour CCD 
cameras for peripheral (120°, horizontally) and foveal (25°, horizontally) views; the eyes can 
perform saccadic eye movements and smooth pursuit of a visual target. The video images 
taken with the cameras are fed into a PC for real-time detection of human faces (by skin- 
colour filtering and template matching) and physical objects such as toys (by colour and 
motion segmentation). The distance to faces and objects can be computed from the disparity 
between the images from the left and right eyes. 

Infanoid has lips and eyebrows to produce various expressions (Fig. 3, left). The lips also move in 
synchronisation with the sound produced by a speech synthesizer. By changing the inclination of 
the lips and the gap between them, the robot can express a variety of emotional states. 
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Fig. 3. Infanoid, the child-like humanoid (right) with an expressive face (left). 

Infanoid hears human voices from microphones positioned at each of its two // ears ,/ and it 
analyses the sound into a sequence of phonemes; it does not have any a priori knowledge of 
language (such as lexicon or grammar). It also recognises salient changes in the fundamental 
frequency to extract emotional contours of human speech. By feeding the output of speech 
analysis into a speech synthesizer, it carries out vocal imitations while sharing attention with the 
interlocutor, which we consider to be a precursor to the primordial phase of language acquisition. 



3.2 Keepon: A Creature-like Robot 

The creature-like robot, Keepon (Fig. 4, left), was designed to engage in emotional and 
attentional exchanges with people, especially babies and toddlers, in the simplest and most 
comprehensive ways. Keepon has a yellow snowman-like body, 120mm tall, made of soft 
silicone rubber. The upper part (the "head") has two eyes, both of which are colour CCD 
cameras with wide-angle lenses (120°, horizontally), and a nose, which is actually a microphone. 
The lower part (the "belly") contains small gimbals and four wires with which the body is 
manipulated like a marionette using four DC motors and circuit boards in the black cylinder 
(Fig. 4, middle, right). Since the body is made of silicone rubber and its interior is relatively 
hollow, Keepon 7 s head and belly deform whenever it changes posture or someone touches it. 




Fig. 4. Keepon' s simple appearance (left) and internal structure (middle/ right). 

The simple body has four degrees of freedom: nodding (tilting) ±40°, shaking (panning) 
±180°, rocking (side-leaning) ±25°, and bobbing (shrinking) with a 15-mm stroke. These four 
degrees of freedom produce two qualitatively different types of actions: 

• Attentive action: (Fig. 5, left) 

Keepon orients towards a certain target in the environment by directing the head 
up/ down and left/ right. It appears to perceive the target. This action includes eye- 
contact and joint attention. 

• Emotive action: (Fig. 5, right) 

Keepon rocks and/ or bobs its body keeping its attention fixed on a certain target. It 
gives the impression of expressing emotions, such as pleasure and excitement, about 
the target of its attention. 
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Note that Keepon can express "what" it perceives and "how" it evaluates the target with 
these two actions. These communicative functions of Keepon 7 s actions can easily be 
understood by human interactants, even babies and toddlers. 



••• 



Fig. 5. Two types of actions: attentive (left) and emotive (right). 

3.3 Modes of Operation 

Infanoid and Keepon are operated in either "automatic" or "manual" modes. In the 
automatic mode, a set of software modules detects the locations of a human face, toys with 
predetermined colour, and moving objects. These locations, together with their likelihood of 
presence, are represented in an "attention map", or a map of saliency or attractiveness. A 
habituation mechanism shifts its attention after being locked onto a strong stimulus for a 
long time. The robots orient their bodies to the most salient target on the attention map; the 
robots' emotional expressions are determined by the type (face/ toy/ motion) and the 
saliency value of the target. Infanoid and Keepon automatically alternate eye-contact and 
joint attention with people in the automatic mode, forming an action loop situated in the 
environment (Figs. 6 & 7). 
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Fig. 6. Infanoid engaging in eye-contact (left) and joint attention (right). 




Fig. 7. Keepon engaging in eye-contact (left) and joint attention (right). 



In the manual mode, a human operator (or a "wizard", usually at a remote PC) controls the 
robots' postural orientations, facial/ bodily expressions, and vocalizations. The operator 
watches video from the on-board cameras and listens to sound from the on-board 
microphone. To perform interactive actions on the robots, the operator uses a mouse to 
select points of interest on the attention map and uses key-strokes that are associated with 
different emotive actions. 
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4. Child-robot Interaction 

We have observed a number of live interactions between Infanoid or Keepon and typically- 
developing children. The children had no prior experience or instructions with the robots. 



4.1 Interaction with Infanoid 

To date, we have observed 15 typically-developing children (from six months to nine years 
of age) interacting with Infanoid (Fig. 8). In these observations, Infanoid ran in the automatic 
mode, in which it alternated between eye-contact and joint attention with pointing. If 
necessary, the operator adjusted the robot's attention, e.g., the orientation of the eyes, the 
head, the arms, and/ or the body. First, each child was seated alone in front of the robot. 
About three to four minutes later, the child's caregiver came in and sat next to the child. 
Interaction continued until the child became tired or bored; on average, each child interacted 
for about 30 minutes. 




Fig. 8. Unfolding interaction between Infanoid and a four-year-old boy: neophobia (left), 
exploration (middle), and interaction phases (right). 

From the series of observations, we found that most of the children, especially those from 
three to seven years of age, demonstrated the following changes in their interaction. 

• Neophobia phase: (Fig. 8, left) 

When the child interacted with the robot alone, for the first three to four minutes, he 
or she looked seriously into the robot's eyes. Even when the robot produced a 
mutual or an aver si ve gaze, the child's eyes were locked onto the robot's eyes. The 
children showed embarrassment and uncertainty about what to do. 

• Exploration phase: (Fig. 8, middle) 

Using his or her caregiver as a secure base, the child next started exploring how the 
robot changed its attention and expressions in response to various interventions, 
such as showing it toys or poking it. When the child elicited an interesting response 
from the robot, he or she often looked referentially at or made commments to the 
mother. Through this exploration, the child discovered that the robot is an 
autonomous agent that shows attention and emotion. 

• Interaction phase: (Fig 8, right) 

The chlid gradually entered into social interactions, where he or she pointed to 

toys or gave them to the robot by putting them in its hand. Verbal interaction also 

started by the child asking questions (e.g., "Which one do you want?", showing 

two toys). 

We assumed that these different phases in the interaction would reflect changes in the 

children's ontological understanding of Infanoid: first as an unknown, ambiguous "moving 

thing", then as an "autonomous system" that had attention and emotions, and finally as a 

"social agent" that deserved to be involved in social interactions, including verbal ones. In 

most cases, these dramatic changes occurred within the first 10 to 15 minutes. 
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4.2 Interaction with Keepon 

We have also observed 25 typically-developing infants in three different age groups, i.e., 
those under one, one-year-olds, and those over two, interacting with Keepon with their 
caregivers (Fig. 9). The robot ran in the manual mode, where a remote operator controlled 
the robot's attentive and emotive expressions manually with the help of video captured by 
the on-board and off -board cameras. The robot usually alternated between eye-contact with 
the infant or the caregiver and joint attention to toys in the environment. When the infant 
demonstrated any meaningful response, such as touching and pointing, the robot made eye- 
contact and showed positive emotions by rocking or bobbing its body. Interaction continued 
until the infants became tired or bored; on average, each infant's dealings lasted about 10 to 
15 minutes. 




Fig. 9. Unfolding interaction between Keepon and a 27-month-old girl: approach (left), 
exploration (middle), and interaction phases (right). Note that those under one stayed in the 
approach phase, one-year-olds reached the exploration phase through the approach phase, 
and those over two reached the interaction phase through the first two phases. 

We found from these observations that infants in each age group showed different styles of 
interaction. 

• Under one year: (Fig. 9, left) 

Interaction was dominated by tactile exploration using the hands and/ or mouth. The 
infants did not pay much attention to the attentive expressions of the robot, but they 
exhibited positive responses, such as laughing or bobbing their bodies, to its emotive 
expressions. 

• One year old: (Fig. 9, middle) 

The infants demonstrated not only tactile exploration, but also awareness of the 
robot's attentive state, sometimes following its attention. Some of the infants 
mimicked its emotive expressions by rocking and bobbing their own bodies. 

• Over two years: (Fig 9, right) 

The infants first carefully observed the robot's behaviour and how caregivers 
interacted with it. Soon the infants started social exploration by showing it toys, 
soothing (by stroking its head), or verbal interactions (such as asking questions). 
The differences between the interactions of each age group reflects differences in their 
ontological understanding of Keepon. The infants first recognised the robot as a "moving 
thing" that induced tactile exploration; then, after observing the robot's responses to 
various environmental disturbances, they recognised that it was an "autonomous system" 
that possesses attention and emotion as an initiator of its own expressive actions. Next, 
they found that the robot's responses, in terms of attention and emotion, had a 
spatiotemporal relationship with what they had done to it; finally, they recognised it as a 
"social agent" with which they could play by exchanging and coordinating their attention 
and emotions. 
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5. Therapeutic and Pedagogical Practices 

Our research fields include a day-care centre for children with developmental disorders, 
especially those with autistic spectrum disorders, and a preschool mainly for typically- 
developing children. At the therapeutic day-care centre, children (mostly two to four 
years old), their parents (usually mothers), and therapists interact with one another, 
sometimes in an unconstrained manner (i.e., individually or within a nuclear 
relationship of child/ mother/ therapist), and sometimes in rather organised group 
activities (e.g., rhythmic play and storytelling). In these dynamically and diversely 
unfolding interactive activities, the children's actions are watched, responded to, and 
gradually situated in the social context of everyday life. At the preschool, a larger 
number of children (three to four years old) interact with one another around Keepon 
with minimum intervention from their teachers. Here, we especially observe how 
various actions and their meanings to Keepon are expressed, exchanged, and shared 
among the children. 

5.1 Keepon in the Playrooms 

A wireless version of Keepon was placed in the two playrooms just like one of the toys on 
the floor. At the therapeutic day-care centre, seven to eight combinations of 
child/ mother/ therapist engaged in the therapeutic sessions (three hours each) in the 
playroom, during which they sporadically interacted with Keepon. During free play (i.e., the 
first hour), children could play with Keepon at any time. During group activities (i.e., the 
following two hours), Keepon was moved to the corner of the playroom so that it did not 
interfere with the activities; however, if a child became bored or stressed by the group 
activities, he or she would be allowed to play with Keepon. 

At the preschool, about 25 children and 3 teachers shared the playroom and sporadically 
played with Keepon in the morning (three hours). Children could play with Keepon at any 
time during free play (i.e., the first 90 minutes). During group activities (i.e., the following 90 
minutes), Keepon was moved to an appropriate position (e.g., on the shelf or on the piano) 
by teachers so that it did not interfere with their activities. 

In the playrooms, Keepon was operated in the manual mode. An operator in another 
room controlled the robot by (1) alternating its gaze between a child's face, the carer's face, 
and sometimes a nearby toy, and by (2) producing a positive emotional response (e.g., 
bobbing its body several times with a "pop, pop, pop" sound) in response to any 
meaningful action (e.g., eye-contact, touch, or vocalisation) performed by the child. We 
manually controlled Keepon because (1) Keepon should wait for a child's spontaneous 
actions, and (2) when the child directs an action, Keepon should respond with appropriate 
timing and manner. 

Throughout the observations, we recorded live interactions between Keepon and the 
children from Keepon' s perspective (Fig. 10). In other words, we recorded all the 
information from the subjective viewpoint of Keepon as the first person of the interaction. 
Strictly speaking, this subjectivity belongs to the operator; however, the interaction is 
mediated by the simple actions that Keepon performs, and every action Keepon performs 
can be reproduced from the log data. Therefore, we may say that Keepon is both 
subjective (i.e., interacting directly with children) and objective media (through which 
anyone can re-experience the interactions), enabling human social communications to be 
studied. 
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Fig. 10. A child seen from Keepon' s subjective viewpoint. 



5.2 Case Studies with Autistic Children 

At the day-care centre, for the past three years (over 95 sessions, or totally 700 child-sessions), 
we have been longitudinally observing a group of children with autism, PDD (pervasive 
developmental disorder), Asperger's syndrome, Down's syndrome, and other developmental 
disorders (Fig. 11). We observed over 30 children in total; some of the children moved in and 
out of the centre during this period. We describe below three typical cases. 




Fig. 11. Keepon in the playroom at the remedial day-care centre. 



Case 1: M — A three-year-old girl with autism 

The first case is a three-year-old girl M with autism. At CA 1:11 (chronological age 1 year 
and 11 months), her MA (mental age) was estimated at 0:10. At CA 3:5, she was diagnosed 
as autism with moderate mental retardation. Here, we describe how the interaction between 
M and Keepon unfolded in 15 sessions over five months (CA 3:9 to 4:1), during which she 
did not exhibit any apparent production of language. 

• From Session 1 (hereafter referred to as SI), M exhibited a strong interest in Keepon, 
but did not get close to it. Through SI to S7, M avoided being looked at directly by 
Keepon (i.e., gaze aversion); however, M gradually approached it from the side and 
looked at it in profile. 

• In S5, after watching a boy put a paper cylinder on Keepon 7 s head, M went to her therapist 
and pulled her by the arm to Keepon, non-verbally asking her to do the same thing. When 
the therapist completed her request, M left Keepon with a look of satisfaction in her face. 
Through S5 to S10, her distance to Keepon gradually decreased to less than 50cm. 

• In the free play of Sll, M touched Keepon 7 s head using a xylophone stick. During the 
group activity, M reached out with her arm to Keepon but did not actually touch it. 
In the intermission of the group activity, M sat in front of Keepon and touched its 
belly with her left hand, as if examining its texture or temperature. 

• After this first touch in Sll, M began acting exploratively with Keepon, such as 
looking into its eyes, waving her hand at it, and listening to its sound. From S12, M 
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vocalised non-words to Keepon, as if she expected some vocal response from it. In 
S13, M put a knitted cap on its head, and then asked her mother to do the same thing. 
In S14, M actually kissed the robot. 
We can see here the emergence of both spontaneous dyadic interactions (Baron-Cohen, 1995; 
Tomasello, 1999), such as touching Keepon with a xylophone stick (Fig. 12, left), and 
interpersonally triggered dyadic interaction, such as putting a paper cylinder on its head 
(Fig. 12, right). The latter especially suggests that M was a good observer of others' behavior, 
although she seldom imitated others even when instructed. Because the boy's action was 
mediated by Keepon and an object (e.g., the paper cylinder) that were of interest to M, it 
would be relatively easy for her to emulate (Tomasello, 1999) the same action and result. 
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Fig. 12. Emergence of dyadic interactions: exploratory actions directed to Keepon (left) and 
interpersonally triggered/ copied actions (right). 

Case 2: N — Another three-year-old girl with autism 

The second case is a three-year-old girl N with autism and moderate mental retardation 
(MA 1:7 at CA 3:1; no apparent language). We observed her interactions with Keepon for 39 
sessions, which lasted for about 18 months (CA 3:4 to 4:8). 

• In SI, N gazed at Keepon for a long time. After observing another child playing with 
Keepon using a toy, N was encouraged to do the same, but did not show any interest 
in doing that. 

• Through S2 to SI 4, N did not pay attention to Keepon, even when she sat next to it. 
However, N often glanced at the robot, when she heard sounds coming from it. 

• In S15, after observing another child place a cap on Keepon's head, N touched 
Keepon with her finger. 

• In SI 6 (after a three-month interval from S15), N came close to Keepon and observed 
its movements. During snack time, N came up to Keepon again and poked its nose, 
to which Keepon responded by bobbing, and N showed surprise and a smile; the 
mothers and therapists in the playroom burst into laughter. During this play, N often 
looked referentially and smiled at her mother and therapist. 

• From SI 7, N often sat in front of Keepon with her mother; sometimes she touched 
Keepon to derive a response. From S20, N started exploring Keepon's abilities by 
walking around it to see if it could follow her. 

• During snack time in S33, N came up to Keepon and started an " imitation game". 
When N performed a movement (bobbing, rocking, or bowing), soon Keepon 
mimicked her; then N made another, and Keepon mimicked her again. Through S33 
to S39, N often played this " imitation game" with Keepon, during which she often 
looked referentially at her mother and therapist. 

We can see especially in S16 and in S33 the emergence of triadic interactions (Baron-Cohen, 1995; 
Tomasello, 1999), where Keepon or its action functioned as a pivot (or a shared topic) for 
interpersonal interactions between N and her mother or therapist (Fig. 13). In those triadic 
interactions, which were spontaneously performed in a playful and relaxed mood, it seemed that 
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N wanted to share with the adults the "wonder" she had experienced with Keepon. Within this 
context, the "wonder" was something that induced smiles, laughter, or other emotive responses 
in herself and her interaction partner. It is also notable that the "imitation play" first observed in 
S33 was unidirectional, in which Keepon was the imitator and N was the model and probably 
the referee; however, this involved reciprocal turn-taking, which is one of the important 
components of social communication. 

t .HI t* z # 

Fig. 13. Emergence of triadic interaction: the child discovers "wonder" in Keepon (left), then 
looks at the partner to share this "wonder" (right). 

Case 3: A three-year-old boy with Asperger's syndrome 

The third case is a boy S with Asperger's syndrome with mild mental retardation 
(MA/ cognition 3:2 and MA/language 4:3 at CA 4:6). Here, we describe the first 15 sessions, 
which lasted for about nine months (CA 3:10 to 4:6). 

• In the first encounter, S violently kicked Keepon and turned it over; then, he showed 
embarrassment, not knowing how to deal with the novel object. 

• From S2, S became gentle with Keepon. Often S scrambled with another child for 
Keepon (S3 and S6), suggesting his desire to possess the robot. In S5, S showed his 
drawing of the both of them to Keepon, saying "This is Pingpong [Keepon]; this is S." 

• In S8, S asked Keepon, "Is this scary?", showing bizarre facial expressions to the 
robot. When an adult stranger approached Keepon, S tried to hide it from her, as if 
he were protecting Keepon. 

• In Sll and S16, when another child behaved violently with Keepon, S often hit or 
pretended to hit the child, as if he were protecting Keepon. 

• During snack time in S14, S was seated next to Keepon. S asked the robot and 
another child if the snacks were "Yummy?". 

• In SI 5, Keepon wore a flu mask. S came up to Keepon and asked "Do you have a 
cough?" a couple of times. When his therapist came in, S informed her of the 
presence of the mask, saying "Here's something". 

In the early stages of interaction, we saw a drastic change in S's attitude toward Keepon. S 
exhibited exceptionally violent behavior towards Keepon in the first encounter. But after S2, S 
demonstrated exceptionally gentle behavior towards Keepon, trying to monopolize and 
sometimes to protect it. His therapist suggested that S usually expressed violent behavior towards 
strangers to whom he did not know how to relate, but he would behave socially after getting used 
towards them. It is noteworthy that S seemed to regard Keepon as a human-like agent that not 
only perceived the environment and evaluated its emotional content, but also understood 
language, regardless of his relatively good cognitive and linguistic capabilities. 

5.3 A Case study with typically-developing children 

Finally, we discuss how a group of 25 typically-developing children in a class of three-year- 
olds (average CA 4:0 throughout the year-long observation) interacted with Keepon in the 
playroom of their preschool (Fig. 14). At around 8:30 a.m., one of the teachers brought 
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Keepon to the playroom and put it on the floor with other toys. In the first 90 minutes, the 
children arrived at the preschool, gradually formed clusters, and played freely with each 
other and with Keepon. In the next 90 minutes, under the guidance of three teachers, the 
children engaged in various group activities, such as singing songs, playing musical 
instruments, and doing paper crafts. Keepon was moved as necessary by the teachers so that 
it did not interfere with the activities; sometimes it sat beside the teacher who was telling a 
story, or sat on the piano watching the children who were singing or dancing. 




Fig. 14. Keepon in the playroom with typically-developing three-year-olds at a preschool. 
(Coutesy of Kyoto Shimbun). 

Throughout the year-long observations (25 sessions of three hours each, over 600 child-sessions), 
we experienced various interactions between the children and Keepon that were qualitatively 
and quantitatively different from what we observed at the remedial day-care centre. Here are 
some anecdotes about what Keepon experienced with the children at the preschool: 

• When Keepon lost its miniature cap crafted for it, a boy TK noticed this and asked Keepon 
"Did you loose your cap?". Keepon nodded, and TK responded with "Endure being 
without your cap", stroking Keepon' s head with an empathetic voice and facial expression. 

• During reading time, a boy TM and a girl NK came up to Keepon and showed it 
picture books one by one. Note that they opened the books in the appropriate 
direction for Keepon to "see" them. 

• Two boys, FS and TA, strongly beat Keepon 7 s head several times, as if demonstrating 
their braveness to each other. Two girls, KT and YT, observing this, approached 
Keepon and checked if it had been damaged; then YT said "Boys are all alike. They 
all hit Keepon", while gently stroking Keepon' s head. 

• A girl YT tried to teach Keepon some words. Showing it the cap, she said, "Say, Bo- 
shi"; then she switched to Keepon 7 s knitted cap and said, "This is a Nitto Bo-shi, that 
you can wear in winter". Note that Keepon could only respond to her by bobbing its 
body with the "pop, pop, pop" sound. 

Especially during free play time (the first 90 minutes), the children showed a wide range of 
spontaneous actions, not only dyadic between a particular child and Keepon, but also n-adic, in 
which Keepon functioned as a pivot of interpersonal play with peers and sometimes with teachers. 
Since the children were generally typically-developing, they often spoke to Keepon, as if they 
believed that it had a "mind". They interpreted Keepon' s responses, although they were merely 
simple gestures and sounds, as having communicative meaning within the interpersonal context. 
We have never observed this with the autistic children, who rarely interacted with peers. 
Compared with the experimental setting (Section 4), where children became bored after 15-minute 
interactions, it is interesting that children in the preschool never lost interest even after 20 sessions. 
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6. Discussion 

6.1 Children's Understanding of the Robots 

We saw in Section 4 that children changed their ontological understanding of Infanoid as the 
interaction unfolded and of Keepon as they grew older: first as a "moving thing ,/ , then as an 
"autonomous system" to explore, and finally as a "social agent" to play with. Interestingly, 
children generally showed a great deal of anxiety and embarrassment towards Infanoid at 
first; however, with Keepon, they spontaneously approached it and started "tasting" its 
texture and motion, and gradually entered into an explorative and social interaction with the 
robot. What created this difference between Infanoid and Keepon? 

We assume that the children first recognize the motions of the arms, hands, eyes, etc., separately. 
Each part of the robotic body emits rich information in its motion; however, it is difficult for 
children to recognize the gestalt of the entirety of these moving parts. The gestalt would be 
"autonomy", "life", or the sense that the robot perceives and acts in the world as we do. In case of 
Infanoid, the children had a difficulty in comprehending the gestalt, which requires (1) effortful 
analysis of the meanings of each moving part and (2) an effortful integration of the meaning into a 
coherent "unity" that all autonomous life would have. Meanwhile, Keepon is completely different 
from humans in terms of its appearance (form), but the simplicity of being able to express only 
attention and simple emotions, combined with the life-like softness of the body, would enable the 
children and infants to intuitively understand the gestalt (Fig. 15). 
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Fig. 15. The different ways that children take to understand the human-like gestalt 
(autonomy or life) in Infanoid and Keepon. 

Understanding of the gestalt, namely, the sense that the robot perceives and acts as we do, 
worked as the motivating basis for children to explore and communicate with it. This would 
be a fundamental prerequisite for any kind of human-robot communication, not only for 
children, but also for people in general. 

6.2 Complexity and Predictability 

Children's ontological understanding of a robot depends also on the complexity of its 
behaviour. What a robot can perceive and how the robot responds to it would change the 
children's "stance" (Dennett, 1987) in interaction with the robot. For example, a robot may 
exhibit periodic actions just like a clockwork toy, reflexive actions in response to some 
specific stimuli, an action situated in the physical environment (e.g., positions of the child or 
toys), or coordinated actions situated within the social environment (e.g., attention and/ or 
emotions of the child). This spectrum of complexity represents the predictability of the 
robot's action, or the action's dependence on internal and external information (Fig. 16). 
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Fig. 16. A design space for interactive robots: the functional and structural complexity 
determines the predictability of the robots' behaviour. 

Although it would be difficult to control their structural complexity, the functional or behavioural 
complexity of robots can easily be manipulated by gradually increasing (or freeing) their degrees 
of freedom and by gradually introducing a dependence on the physical and social situations to 
their behaviour. Manipulating the structural and, especially, the functional complexity, we can 
tune the predictability of a robot's behaviour to each child's cognitive style and developmental 
stage, providing a "zone of proximal development" (Vygotsky, 1934/1962). When a child 
encounters a robot whose behaviour has an appropriate predictability for that particular child, he 
or she should approach the robot in a relaxed mood and spontaneously start explorative 
interactions in a playful mood. From this, the child might subsequently learn to predict and 
control the robot's behaviour in terms of its dependence on physical and social situations. 

6.3 Facilitating Social Interactions 

When the complexity of a robot is appropriate for a particular child, the robot's behaviour 
attracts his or her attention and elicits various actions from the child (Fig 17, left). As we saw 
from the case studies (Section 5), the children enjoyed the dyadic interaction with the robot 
with a sense of security and curiosity, in which they gradually learned the meanings of the 
robot's responses. The way this dyadic interaction unfolds varies from one child to another, 
but it is noteworthy that every individual child, either spontaneously or with minimum 
intervention by a therapist or mother, builds a relationship with the robot. 
The robot also attracts a child's attention to another child who is interacting with it. As we 
witnessed in the case with the girl M, each child curiously observed how others were acting on the 
robot and what responses they induced from it. For the child observer, the interactions between 
the robot and the other child were comprehensive, because of its structural simplicity and 
functional predictability. Especially when the other child induced a novel, interesting response 
from the robot, the observers would also feel pleasure and excitement (Fig. 17, middle). 



Fig. 17. Facilitating children's interpersonal interaction: a child finds a "wow" in the 
interaction with the robot (left), observes another child's "wow" (middle), and relates the 
self's "wow" and other's "wow" using the robot as the pivot. 
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A child is surprised at the robot's interesting response and observes others also being surprised 
at the same or similar response from the robot. These two "wows" are qualitatively equivalent, 
since both come from a simple robot with moderate predictability. Here, we can see that an 
interactive robot has the potential to serve as a pivot for relating the child's own "wow" with 
others' "wow" (Fig. 17, right). As we saw in the case studies, when the girl N induced a 
surprising response from Keepon, probably with the help of the therapists, who exaggeratedly 
reacted to her discovery by bursting into laughter, N checked with her mother for the same 
"wow" as if she were saying "Mom, it's interesting/ Did you see that?". 

7. Conclusion 

We have presented in this paper design principles of interactive robots for inducing in children 
various spontaneous interactions based on their ontological understanding of the robots. We 
have built an upper-torso humanoid, Infanoid, and a creature-like robot, Keepon. Both can 
engage in eye-contact and joint attention with human interactants, though which the robots and 
humans can exchange emotional and attentive states. In our psychological experiments on child- 
robot interaction, we found that children regarded Infanoid as a "moving thing", an 
"autonomous system", and a "social agent" as the interaction unfolded during the course of 30 
minutes sessions. With Keepon, children deepened their interaction as with Infanoid; those 
under one, one-year-olds, those over two reached the understanding of the robot as a "moving 
thing", an "autonomous system", and a "social agent", respectively. 

We then carried out field studies at a remedial day-care centre and at a preschool, where we 
longitudinally observed interactions between children and Keepon in rather unconstrained 
everyday situations. These field studies suggest the following: 

• Children, even those with difficulty in interpersonal communication (e.g., those 
with autism), were able to approach Keepon with a sense of curiosity and security. 
This was probably because the robot seemed to be neither a complex human nor a 
simple toy. 

• Most typically-developing children and some with developmental disorders 
extended their dyadic interactions with Keepon into triadic interpersonal ones, 
where they tried to share the pleasure and surprise they found in Keepon with others, 
such as their caregivers. 

• Each child exhibited a different style of interaction that changed over time, which 
would tell us a "story" about his or her personality and developmental profile. These 
unique tendencies cannot be thoroughly explained by a diagnostic label such as 
"autism" or a broad psychological term such as "introverted". 

The "story" of each child has been accumulated as video data, which is being utilized by 
therapists, psychiatrists, and paediatricians at the day-care centre for planning their 
therapeutic intervention, and by teachers at the preschool for improving their educational 
services. We also provided the video data to parents with the hope that it may positively 
influence their own child-care. 

Although we conducted our field studies using the manual mode, where a human operator 
tele-operated the robots, our observations demonstrate that interactive robots with 
appropriate structural and functional complexity can facilitate children's social interactions 
with robots, peers, and carers. In creating robots that can autonomously socially interact 
with people, we are still missing a wide range of robotic and AI technologies; however, for 
children, especially those with developmental disorders, current technology for interactive 
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robots, even when tele-operated, can certainly be applied to facilitating their social 
interaction and its development. 
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1. Introduction 

Robot technologies have been developed for robots used in special environments such as 
industrial robots, maintenance robots for nuclear plants and robots used in space. Recently, 
the scope of robot applications has been expanded to include the medical and welfare fields, 
maintenance and patrolling of buildings, and home-use robots. Although some robots have 
already been developed, work on practical robots for this field has only recently begun. 
Toshiba's development of robot technologies has also reflected this trend (Matsuhira & Ogawa, 
2004). Robots to support human life in the home, at public facilities, or outside of factories are 
called "life support robots' 7 . In this field, it is essential that robots understand the human intent 
and interact with humans and the environment. Thus, human-centric technologies are 
important not only for robots but also for all machines that humans use. On the other hand, 
Toshiba defined "humancentric technology 77 as technology that contributes to human life and 
society. Robots are representative of humancentric technology as shown in Fig. 1. Such robots 
will interact with humans by the integration of hearing, vision, and motion. 
Toshiba has already developed a conceptual model of a robotic information home appliance 
" ApriAlpha™" to provide an interface between human and networked appliances (Yoshimi, 
et al., 2004). The robot is considered to be applicable to various fields. Based on ApriAlpha™, 
two types of robots offering improved performance, namely, ApriAlpha™ V3 and 
ApriAttenda™, are being developed. ApriAlpha™ V3 features an enhanced omnidirectional 
auditory function and ApriAttenda™ is a person-following robot. These two robots have 
been developed as part of the next-generation robot project of the New Energy and Industrial 
Technology Development Organization (NEDO) and exhibited to verify the performance in 
the prototype robot exhibition in June at Aichi EXPO 2005. Toshiba and Tokyo University of 
Science collaborated on the development of ApriAttenda™. 

The remainder of this paper is organized as follows. Chapter 2 explains the problems of life 
support robots. Chapter 3 explains ApriAlpha™ in terms of its performance as a robotic 
information appliance and the network communication technology it employs. Chapter 4 
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explains the sharp ear robot ApriAlpha™ V3, which features an enhanced auditory function. 
Chapter 5 explains the person-following robot ApriAttenda™, which is designed to 
accompany a person and serve him/her anywhere. Chapter 6 explains the navigation 
technology implemented in ApriAttenda™ for map building and performing 
self-localization. Chapter 7 explains the importance of environmental design, including 
information and physical environments with universal design, to realize working robots in 
daily life. 
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Fig. 1. Concept of human and robots with humancentric technology. 



2. Subjects of Life Support Robots 

In recent years, Japan and certain other countries have experienced a declining birthrate and 
a growing population of elderly people, the widespread introduction of information 
technology, and a growing preoccupation with security, including in the home. In particular, 
it is forecast that the numbers of skilled workers will decrease from 2007 onward in Japan. 
These developments pose difficult problems, which robot technologies are expected to 
ameliorate provided reliable and safe robotic technologies become available for applications 
in everyday life. However, there are many problems to be solved. In industrial plants and 
nuclear facilities, suitable environments are prepared in advance for robots. Parts are 
designed for easy handling and easy assembly procedures for robots. Working environments 
and assembling parts are prepared for robots. On the other hand, in the field of everyday life, 
the following problems remain because of the variation of users' requirements. 

a) Working environment and required tasks, 

b) Image processing environment including background scene and lighting, and 

c) Auditory processing environment including background noise such as TV are not 
constant. 
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Furthermore, robots are desired that can be used easily by anybody. To solve these problems, 
it is necessary to develop simple robots first for limited environments and tasks. Next, 
multifunctional robots should be developed by defining tasks step by step. And 
environmental design and knowledge embedded in the environment will be collaborated 
with robots, as necessary. 

In daily life, not only the technologies for motion control of manipulation and mobility to 
perform the actual tasks, but also human machine interface technologies are important, such 
as for interaction between humans and robots by image and voice. A robot cannot perform 
services unless it communicates with a human to ascertain where he/she is and what he/she 
wants. Furthermore, it is natural and reliable for a robot to accompany the human when the 
robot performs services. Accordingly, a robot with omnidirectional auditory function and a 
robot with visual tracking function for a specified person have been developed as human 
interface technology. 

Since a robot is a system, even if the image or voice processing function works well alone, the 
overall performance of the robot may be insufficient. Seamless integration of all functions, 
not only vision and auditory functions but also motion, is required. Regarding systemization, 
so far, the open robot controller architecture has been adopted to integrate auditory, vision, 
and motion functions as shown in Fig. 1. Robots working in everyday life need integration of 
vision, hearing, and motion. Vision and hearing are the interface functions enabling the robot 
to recognize the person and the commands. The voice and sound source position detected by 
hearing, and environment, obstacles, and human faces will be detected by vision, and the 
inputs' information is considered in order to plan for the robot motion. For that reason, we 
developed the auditory function and the person-following function and navigation function 
using the vision system as core technologies of human-friendly robots as described below. 
First, the basic function of Apr i Alpha™ is explained. 

3. Robotic Information Appliance "ApriAlpha™" with Network Ability 

ApriAlpha™ is a conceptual model of a robotic information appliance. It recognizes the face and 
voice of the user and replies to the user by voice, and moves by wheels in the home. The robot was 
developed to perform the functions of house sitting and intelligent remote control for home 
appliances. Here, ApriAlpha™ means an advanced personal robotic appliance type alpha. 

3.1 Open robot controller architecture 

For ApriAlph™, the framework of open robot controller architecture was adopted, 
integrating various robot technologies with a distributed object technology based robot 
controller that we are currently developing (Ozaki, 2003). An expandable and easy-to-use 
robot controller can be realized. The distributed object technology can deal easily with 
various objects for various functions distributed on a network. When executing the 
desired object, it is unnecessary to be conscious of the object's place on the network. The 
desired object can be executed by issuing a command message in a standard format. 
Various robot technologies and/ or interfaces with peripheral equipment are included in 
the system using HORB developed by the National Institute of Advanced Industrial 
Science and Technology (AIST) (http://horb.aist.go.jp/). Various modules of 
ApriAlpha™ are easily combined as shown in Fig. 2. By upgrading these modules or 
adding new modules, the performance of the robot will be improved or a new robot will 
be realized. 
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3.2 Networked robot 

Fig. 3 shows the cooperation system with networked appliances (Matsuhira et al., 2005), 
(Ueno et al., 2004). A refrigerator, an air conditioner, and a cooking range are connected to 
the LAN through the access point of Bluetooth™. The server of a question-answer system is 
also connected to the LAN with the UPnP™ protocol. These appliances are recognized 
automatically by UPnP™ technology. The program of the robot controller can be described 
bilaterally in this framework, independent of the category of the apparatus, which includes 
robots or home appliances. Depending on the user's requests, the robot provides the latest 
news via the internet, controls the temperature of an air conditioner via the LAN, checks the 
ingredients in the refrigerator, displays a recipe, and sets the cooking range appropriately. 

(1) Agent Technology 

Flipcast™, developed by Toshiba, is a lightweight agent technology. The application of 
searching news requested by the user has been developed using Flipcast™. When the user 
asks, "What is today's news?" or "Tell me the sports news." to ApriAlpha™, ApriAlpha™ 
searches the internet in order to provide the latest news corresponding to the categories of 
economy, general news, and international news. The script using Flipcast™ combines news 
acquisition and an analysis function implemented on the server of the National Institute of 
Informatics (Nil) and the voice recognition and synthesis functions of ApriAlpha™. Since this 
application is very simple, the script can be made easily for information search and 
information is acquired on the internet depending on the user's request. 

(2) Question-answer system 

Multi-modal help, using a question-answer system developed to support the user, 
enabling him/her to make full use of home electric appliances and AV equipment, is 
connected to the LAN. This system can offer information suitable for the user, searching 
technology using multi-modal knowledge content consisting of graphical images, audio 
assist, and text images. 

(3) Cooperation with Networked Home Appliances 

Numerous communication protocols for home networks have already been proposed 

such as ECHONET™ for home electric appliances, IEEE1394 for AV equipment, wired 

and wireless LAN, and telephone line. In particular, ECHONET™ can connect home 

appliances in a home network and operate appliances and manage their conditions. It is 

too difficult for a home robot to support all of these protocols used in home appliances. A 

new framework has been developed for cooperation between a home robot and 

networked appliances (Morioka et al., 2004). Within this framework, all of the protocols 

used in the home appliances concerned are combined and structured with UPnP™ 

technology. Using the UPnP™ protocol, all features of the original protocols are provided 

via the interface of the framework. 

In Fig. 3, when the user asks ApriAlpha™, "Is there any news?", ApriAlpha™ searches the 

latest news and reads it out by using the agent function. When the user says, "I feel rather hot.", 

ApriAlpha™ controls the temperature of the air conditioner automatically. When the user asks, 

"What is in the refrigerator? Tell me a recipe suitable for the ingredients.", ApriAlpha™ 

displays a suitable recipe using the question-answer system for recipes. The user can ask 

questions about the recipe during preparation and cooking. Furthermore, ApriAlpha™ can 

input the recipe to the cooking range and notify the user when the cooking is finished. In the 

future, by engaging in simple conversations with robots that support their household affairs, 

people will be able to enjoy comfort and peace of mind, having been liberated from the 

necessity of executing troublesome operations with remote-control devices. 
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TM 



4. Sharp Ear Robot "ApriAlpha" vl V3" 

4.1 The purpose of the sharp ear robot 

In general, various noises exist in the home besides the orders spoken by the user to a robot. These 
include speech, the sound of the TV and other sound sources shown in Fig. 4. Thus, home robots 
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require a function enabling them to hear each voice from any direction and recognize the contents 
among the noises encountered in daily life. Therefore, research on an auditory function to recognize 
the voices around a robot has been performed actively (Nakadai et al., 2003), (Brandstein, 1999), 
(Asano, 2004), (Brandstein & Ward, 2001). We have realized a high-performance auditory function 
for sound source localization by the development of signal processing technology to detect a voice 
from any direction around a robot and to assume the voice source direction. 



Wa-! - 



Can you check 
for e-mail ? 





Fig. 4. Images of Apri Alpha™ in a living room. 

4.2 Auditory signal processing for omnidirectional auditory function 

Fig. 5 shows the block diagram of auditory signal processing to perform the omnidirectional 
auditory function. Two pairs of microphones from among the multiple microphones are 
selected and perform an auditory signal processing. 

First, to assume the sound source direction, AT, which is the arrival time differences between two 
microphones, is used. If it is assumed that the wave from the sound source to the microphone is the 
plane wave, gained AT denotes the distance between the sound source and the two microphones. 
The angle between the microphone and the sound source direction is given as follows: 

6=sin-i(AT/d) ; (4-1) 

here, d is the distance between two microphones. Auditory signals are transformed by fast Fourier 
transformation (FFT) to phase difference and intensity data in the frequency region. Then, the 
number of the sound sources and their directions at each moment are assumed by the phase 
difference analysis, and the voices are detected as the time series of voice stream for each sound 
source. Here, Hough transformation was used to assume the sound source directions (Okazaki, 
2000). Next, by matching the voice streams whose frequency ingredient and voice length are 
similar, the space position of a sound source is presumed. When the sound source positions are 
identified, noise reduced voice stream abstraction, by adapted array processing, is possible and 
each voice stream is recognized by the voice recognition engines. Here, array processing and 
recognition engines, which have already been developed, are used (Amada, 2004), 
(http://www3.toshiba.co.jp/pc/lalavoice/index_j.htm (in Japanese)). Thus, an auditory function 
that recognizes voices from any direction has been realized. 



4.3 Developed ApriAlpha™ V3 

To verify the developed auditory function, ApriAlpha™ V3 was developed based on 
ApriAlpha™ as shown in Fig. 6 and Table 1. It incorporates a prototype auditory signal 
processing board with an 8 ch parallel input interface. In the robot, auditory signals are input 
from six microphones mounted on the surface of the robot's body. Owing to smooth fusion 
with auditory function and mobile function, the robot can change direction quickly 
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whenever a voice is heard and approach the user. Fig. 7 shows the voice data processing 
result. This is an example of the result which detected the voice stream when two persons 
speak simultaneously from the different directions. One person stands at 20 degrees left side 
of the robot and the other stands at 40 degrees right side. From the directions where persons 
stand, voice streams were correctly extracted. 

The developed robot performed a demonstration and verified the capability of omnidirectional 
auditory functions in an actual noisy exhibition hall at Aichi EXPO both in the "NEDO prototype 
robot exhibition" and the "robot exhibition at the Robot Station". In the prototype robot 
exhibition shown in Fig. 8, a demonstration lasting about 10 minutes was performed 3 times per 
day, and two kinds of demonstrations were carried out with respect to a robot located at the 
center of a circular table with a diameter of 2 meters. In each demonstration, three people called 
the robot continuously from different positions around the table. One is a demonstration of 
household-electric-appliance control and information retrieval. Speaking in turn, the people 
order the robot to turn the air-conditioner and the electric light ON/ OFF, and order the robot to 
retrieve the weather information from the Internet via wireless LAN and provide it. The robot 
could hear the orders and execute them without interruption. In the other demonstration, a robot 
is the chef of a sushi bar. The robot turns to the person who orders sushi from the circumference 
of a small stage, and recognizes the directions and responds by uttering the name of the type of 
sushi. Here, the number of voice commands is limited to about 10 words/ phrases in order to 
verify the sound source localization. In the exhibition hall, the noise level at the circumference 
reached 80-85 dB at the start of the demonstration on the main stage of the Morizo-Kiccoro 
Messe. However, it was verified at our demonstration stage that the developed auditory 
function could work when the noise level at the circumference was about 80 dB. Moreover, 
Apri Alpha™ V3 could respond to a visitor's voice at our local demonstration. To improve the 
auditory function, vision will be combined to search for the sound source localization. For example, 
the robot turns to the direction from where sound was detected to search for a face, and if a face is 
detected the localization performance is improved. When a voice command is not detected clearly, 
the robot can move to the position where the voice is detected more easily. So, sensing and motion 
should be combined to execute the task, rather than concentrating on the development of a 
dedicated sensing method. Of course, from the viewpoint of practical use, other methods of 
inputting commands should be considered, such as a touch panel. 
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Fig. 5. Block diagram of auditory signal processing. 
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Fig. 6. Sharp ear robot ApriAlpha™ V3. 
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Fig. 7. Data processing results of ApriAlpha™ V3. 




Fig. 8. Demonstration at Aichi EXPO. 

ApriAlpha™ can recognize sequential orders from surrounding users. 



Dimensions 


Diameter: 380 mm 
Height: 430 mm 


Weight 


About 10 kg 


User Interface 


Microphone: 6 
Speaker: 2 
CCD Camera: 2 
LCD with a touch panel 


Motion 


2 Independent drive wheels 
Face: rotate joint 
Eye: pan and tilt 


Communication 


Wireless LAN: IEEE802. 11 a/b 


Power source 


Lithium-ion battery: about 2 hours continuous motion 



Table 1. Specifications of ApriAlpha™ V3. 



Research and Development for Life Support Robots that Coexist in Harmony with People 



295 



5. Person-Following Robot "ApriAttenda™" 

5.1 The purpose of the person-following robot 

It is necessary for a robot working in the environment of everyday life not only to move to 
a specified place following a predetermined trajectory, but also to move with flexibility. To 
perform the motion, a person-following function that involves finding the specified person 
and following him/her is essential. Such a robot is expected to take care of an infant and or 
an elderly person, control home electric appliances, offer information services such as the 
news, and carry baggage as it follows the user in a shopping center as illustrated in Fig. 9. 
The robot can support everyday life from the viewpoints of safety, security and 
practicality. 





Babysitting 

Watch over the elderly 

Fig. 9. Image of ApriAttenda™ used in everyday life. 



Carry bags 



5.2 Image processing of person-following robot 

The basic functions involved in following a person are as follows: finding the specified 
person, following at his/her pace, avoiding obstacles, and resuming contacting when the 
robot misses him/her. Person-following robots developed until now use various types of 
cameras for detecting a target person, and some of them use other sensors (Schlegel et al., 
1998), (Sidenbladh et al, 1999), (Kwon et al, 2005), (Cielniak et al, 2005), (Kobilarov et al, 
2006). Our newly developed robot uses a stereo vision system. The combination of the vision 
system and the motion control system makes it possible to realize a robust person-following 
function. The control system from the viewpoints of function modules is shown in Fig. 10. 
For the person following function, Target Detection Module and Motion Control Module, 
were constructed. Two camera images of the target person including cluttered backgrounds 
are captured concurrently by the newly developed versatile multimedia front-end processing 
board (MFeP) (Sato et al., 2005), and sent to the Target Detection Module. At the Target 
Detection Module, the target person is detected by the newly developed image processing 
algorithm, and the result (distance and direction data of the target person from the robot) is 
sent to the Motion Control Module through the network. At the Motion Control Module, the 
two wheels and the head of the robot are controlled to follow the target person smoothly. The 
Target Detection Module is running on Windows PC and the Motion Control Module is 
running on Linux PC, because the Windows PC has many image processing applications, 
and the robot motion control requires real-time processing. The frame rate of the image 
processing system is about 15 fps, and the control cycle of the motion control system is 1kHz. 
Regarding ApriAttenda™' s systemization, the open robot controller architecture as shown in 
Chapter 3 has been adopted to easily integrate the Target Detection Module and the Motion 
Control Module. 
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To find the person, a shape of a human back and shoulder for visual tracking of the target 
person was used (Hirai & Mizoguchi, 2005). A new algorithm has been developed to recognize 
and abstract the region of the person from an image containing a complicated scene such as that 
shown in Fig. 11. The point at which the texture changes is abstracted automatically as a feature 
point. The distance from each feature point is measured by the stereo vision system, and then 
the person area is detected by the distribution of the distance and the history of the motion of 
feature points. Furthermore, by combining the information of the color and texture of the 
clothes the target person wears, the person area is identified without fail. A robust method 
against the lighting and change of scene has been developed by utilizing these variable 
information and importing and updating refresh feature points of the person detection area. 
In the following motion, the data captured by an ultrasonic sensor is imported during the 
person detection processing in parallel. When an obstacle is found by the ultrasonic sensor on a 
trajectory, the robot continues to follow the person by the vision sensor while the robot's body 
avoids the obstacle. Furthermore, in the case that the robot fails to detect and follow the person, 
searching motion starts to increase the searching area. A re-finding function is implemented 
that matches between the registered image data and the input image data of the clothes' texture. 
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Fig. 10. Control system of ApriAttenda™ 
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(a) Processing image. (b) Finding the person. 

Fig. 11. Image processing algorithm of ApriAttenda™ 
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5.3 Developed ApriAttenda 

Fig. 12 shows a scene in which ApriAttenda™ is following a person. The specifications of 
ApriAttenda™ are shown in Table 2. The robot is 450 mm in diameter, 900 mm in height, and 30 
kg in weight, and the robot shape is designed attractive and unintrusive. The robot finds the 
person by means of two CCD cameras on its head. Motion control with a newly developed inertia 
absorbing mechanism in the driving part realizes stable and smooth person-following motion. 
At Aichi Expo, a robot could follow a person who walked freely in the demonstration space in 
front of many spectators as shown in Fig. 13. ApriAttenda™ calculates the distance between the 
target person and itself using stereo vision and follows him/her with the appropriate speed to 
keep the distance constant. Sometimes the robot failed to find the person, but once he/she 
moved into sight of the robot, the robot followed him/her again. In another demonstration, the 
robot could follow a child with a stuffed mascot "Kiccoro". The mascot was the target to be 
followed by the robot because anyone could join the robot-following demonstration by holding 
the mascot. This demonstration was also successful in the case that spectators were closer to the 
robot as shown in Fig. 13(b). Here, this robot verified the performance of person finding and 
tracking only by vision. But if the robot can hear the voice and recognize the face, or move to 
the position where it can search the voice and face more easily, the reliability of finding the 
person is drastically increased. So, the system integration in robots to perform the task robustly 
is very important as described in Chapter 2. 




Fig. 12. Person-following robot ApriAttenda™. 



Dimensions 


Diameter: 450 mm 
Height: 900 mm 


Weight 


About 30 kg 


User Interface 


Microphone: 2 
Speaker: 2 
CCD camera: 2 
LCD with a touch panel 


Motion 


2 Independent drive wheels 
Face: rotate 2 Joints 
Eye: pan and tilt 


Communication 


Wireless LAN: IEEE802. 11 a/b 


Power source 


Lithium-ion battery about 2 hour continuous motion 



Table 2. Specifications of ApriAttenda™. 
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(a) Demonstration stage (b) Searching for the mascot. 

Fig. 13. Demonstration of ApriAttenda™ at Aichi EXPO. 

6. Navigation of the Mobile Robot 

Although ApriAttenda™ has a function for person searching and following, if the robot is 
ordered to watch or bring something, a guiding function is necessary to enable the robot to 
move freely in a house or any other building. For such a function, the map of a building is 
required. To build the map, a service-person or a user must input the map manually, teach 
the route using by a remote control device, or use the following function of the robot as 
described above. Target markers on the selected places and the function of local obstacle 
avoidance with ultrasonic sensors and so on are used for the robot to move to the desired 
position autonomously. In any event, target positions are input in advance. With regard to 
building a map, a great deal of research has been done on simultaneous localization and 
mapping (SLAM). Recently, the field of robotic SLAM is dominated by probabilistic 
techniques (Thrun, 2002). In this chapter, a novel map building and localization approach is 
explained. The map building stage is based on shape similarity comparison, which was 
originally developed for computer graphics applications. And the localization stage is based 
on an improved Monte Carlo localization. In the following parts of this Chapter, map 
building and localization are discussed, respectively. The experimental results are given 
based on the person-following robot ApriAttenda™ (Zhu et al., 2005). 

6.1 Map building based on shape similarity 

To acquire a map, the robot must memorize the perceived objects and features the robot has 
passed by, then merge the corresponding objects in consecutive scans of the local 
environment. Here in our approach, the information from LRF (Laser Range Finder) sensor 
on the robot and odometry is used to build a global map. 



(1) Map building process 

In the developed algorithm, the global map is built iteratively as the robot moves. We denote 
the scan and the global map at time t by S t and G t , respectively. Each scan S t and global map 
G t is composed of polylines. To create the global map G, we start with the first global map Gi 
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being equal to the first scan Si. Assuming at time t-1 the global map G t-i has been created, Gt-i 
and S t are used to create G t in the following steps: 

a) St is converted into a cyclic, ordered vector of polylines. At this stage, noise and 
unreliable information in the scan reading should be excluded. 

b) A virtual scan VSt is extracted from the global map Gt-i. VSt is the part of the global 
map, which is assumed to be seen by the robot at position t. 

c) Sampling points from VSt and S t , then searching the corresponding points between 
VS t and S t . 

d) Repeatedly transforming St with different translation and rotation offset. By calculating the 
shape similarity value between VS t and St, an optimal transformation matrix can be found, 
which will assign a minimal value to the shape similarity between two scans. 

e) Finally, the aligned St is merged into G t -i to create G t . 

Repeating the above steps iteratively for each new scan, we are able to create a global map 
that remembers all the objects and features along the path the robot has traveled. 
Here, the first step in this approach is excluding the noise from the scan, and grouping the 
valid scan points into cyclic ordered polylines. To make the representation more compact 
and to cancel noise, we employ a technique called DCE (discrete curve evolution) (Latecki & 
Lakamper, 1999) to achieve this goal. 

(2) Matching map by shape similarity comparison 

Before aligning a scan to the global map, not all the features in the scan and the global map 
should be compared. Here to save the computation cost, a virtual scan VSt from the global 
map G t -i is generated to calibrate the scan St. Since VS t and V t may have some information, 
which the other doesn't have, preprocessing is necessary to find out the corresponding points 
before the shape similarity comparison. Since we attempt to use the known information, 
which is assumed to be correct, to align the new scan, only the points with corresponding 
points will be used in shape similarity comparison. The points in candidate set VS t are 
treated in the same way as well. Matching scans against the global map within the context of 
a shape-based representation is naturally based on shape matching. In order to find out how 
similar two shapes are to each other, we must know how different they are to each other. 
For comparing two 3D models, the most commonly used error metric is L2 norm. Here we 
would like to formulate shape-based analogs of L2 norm. If P(Si) and P(S2) denote all the 
points in shape Si and S2, we can select two sets x l c P(S X ) and x 2 c P(S 2 ) containing k Y and k 2 
sample points, respectively. Thus, the sampled error metric between these two shapes 
E avg (S l ,S 2 ) is the average square distance from the sample points in Xi and X2 to shape S2 

and Si. In the application for comparing two scans, not all the points in the candidate point 
set are treated in the same way; a larger penalty coefficient is given for the feature points in 
similarity comparison, since mismatching of the feature points will result in a worse 
alignment for two scans. We let sf be the scan transformed from St by (£x,sy,£0) . By 
iteratively transforming St with different (£x,ey,€0) , an optimal transformation vector 
(Ax t , Ay t , A0 t ) could be found, which will assign a minimal value to the shape error metric. 
Therefore, it could be described as an optimization problem. 

Goal: mm{E avg (VS t ,sJ)) 

Where: sj = T(S t , ex, ey, £0) 



300 Mobile Robots, Towards New Applications 

Constrains: min(x) < ex < max(x) 

min(^) <ey < max(^) 
min(<9) < e6 < max(<9) (6-1) 

The odometry error of the robot at time t will be the optimal solution of the above 
optimization problem. 

(3) Merging maps 

Merging is the task of combining similar line segments taken from the sensor reading S t and the 
previous global map Gt-i to form a new global map Gf. During the merging process, we need to 
accurately identify objects in each scan, look for their corresponding objects in the existing global 
map, update the position of the object, remove moving objects, and connect separated objects into 
a single object whenever it is possible. Fig. 14 shows the map building result in Toshiba Science 
Museum, 1 st floor hall (14 m x 18 m). The robot navigation trace from the odometry is displayed by 
the red line and the calibrated robot trace is displayed by the blue line. 

6.2 Localization problems 

The localization problems are of many different types (Thrun, et al., 2001). In the case of 
position tracking in Fig. 15, the initial robot position is known, and the problem is to 
compensate incremental errors in a robot's odometry. In a global localization problem, robot 
is to determine the position from the scratch. In the case of a global localization problem, the 
error in the robot's estimate cannot be assumed to be small. In the case of a kidnapped robot 
condition, a well-localized robot is moved to some other place without the robot's 
knowledge. The kidnapped robot problem is often used to test a robot's ability to recover 
from catastrophic localization failures. Unlike other algorithms, the Monte Carlo localization 
(MCL) approach solves a position tracking problem, a global localization problem and a 
kidnapped robot problem in a robust and efficient way. 

(1) Monte Carlo localization approach 

MCL uses Bayes filter to estimate the posterior distribution of robot poses based on sensor 
data in a recursive manner. According to Bayes filter, the future data is independent of past 
data if we have knowledge of the current state — an assumption typically referred to as the 
Markov assumption. In this case, the dynamical system is a mobile robot and its environment, 
the state is the robot's pose therein (a two-dimensional Cartesian space and the robot's 
heading direction), and measurements may include range measurements, camera images, 
and odometry readings. 

If x denotes the state, x t is the state at time t, and do.j denotes the data starting at time up to time t . 
Then, probability density of robot position is p(x t ) = p(x t | do.j). For mobile robots, perceptual data o 
(for observation) such as laser range measurements, and odometry data a (for action), carry 
information about robot motion. Thus probability density of robot position becomes 

p(x t ) = p(x t | o t/ a t -i, ot-i, a t -2, . . • ,oo) (6-2) 

where at-i refer to the odometry reading that measures the motion that occurred in the time 
interval [t-1; t\. 

If initial knowledge is not available, it is typically initialized by a uniform distribution over the 
state space. To derive a recursive update equation, Eq.(6-2) can be transformed by Bayes rule 
and Markov assumption to (Dellaert et al.,1999). 
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p(x t ) = np(o t | x t )\p(x t | xt-i, a t -i)p(xt-i)dx t -i (6-3) 

This equation is the recursive update equation in Bayes filter. Thus if we know the state of the 
robot at previous step, action or movement after the previous step and the observation or 
measurement information at the present state, we can easily determine the present state 
without knowing all the previous steps taken by the robot. Together with the initial belief, it 
defines a recursive estimator for the state of a partially observable system. This equation is 
the basis for MCL algorithms used in this work. 

As mentioned above, to implement (6-3), two conditional densities are required: the 
probability p(x t \ Xt-i, at-\), which we will call next state density or simply motion model, and the 
density p(o t \ x t ), which we will call perceptual model or sensor model. By simplifying, we can 
express these models as p(x' \ x, a), and p(o \ x), respectively. 

(2) Probabilistic models for localization 

The motion model, p(x ' \ x, a), is a probabilistic generalization of robot kinematics. In this 
work, each pose comprises a robot's two-dimensional Cartesian coordinates and its heading 
direction (orientation, bearing). The value of a may be an odometry reading or a control 
command, both of which characterize the change of pose. The conventional kinematic 
equations, however, describe only the expected pose x ' that the ideal, noise-free robot would 
attain starting at x, and after moving as specified by a. Of course, physical robot motion is 
erroneous; thus, the pose x' is uncertain. To account for this inherent uncertainty, the 
probabilistic motion model p(x ' \ x, a) describes a posterior density of x '. In MCL the margin 
of uncertainty depends on the overall motion. For the MCL algorithm we need a sampling 
model o(p(x' \ x, a) that accepts x and a as an input and generates random poses x ' distributed 
according top (x' \ x, a). 

For the perceptual model or measurement model, p(p \ x), mobile robots commonly use 
range finders, such as ultrasonic transducers (sonar sensors), laser range finders, and 
camera images. A laser range finder and camera data are considered for localization in this 
work. In this work, first, the value of mean and standard deviation a noise-free sensor 
would generate is determined. Then the marker distance from the current robot position is 
calculated. Finally, the information of laser beams and marker distance is integrated into a 
single density value. 

As noticed by other authors (Thrun et al., 2001), (Dellaert et al., 1999), the basic particle filter 
or MCL performs poorly if the number of particles is small, and if the number of particles is 
large, the processing time will increase and real-time or near real-time application will be 
impossible. Considering these problems, we kept the number of the particles to 100 in this 



(3) Experimental studies 

In this work, we took actual data with localization error from the robot movement in the 
Toshiba Science Museum in Fig. 15(a). The experimental scene is shown in Fig. 16. We 
used a similar map of the museum in this work. The actual map is more complex than the 
map we used in this simulation work. Generally, MCL algorithm works better with a 
comparatively more complex map such as that in Fig. 15(a) rather than with a simplified 
one, because in the simplified map there are many similar places and mean and standard 
deviations of LRF scan lines at these positions are close. In actual maps fewer places are 
similar and the LRF scan data are also different for different places. So mean and standard 
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deviations will also vary at different places and as a result localization works better in 
such a situation. 

For localization, the robot has to be sure whether it should start position tracking steps or 
kidnapped condition recovery steps. If the ideal probability (calculated using LRF scan 
mean, standard deviation and the distance from the specific marker based on the map) 
and the actual probability (LRF and marker information collected by robot) differ 
significantly (e.g. more than 20%), the robot will start kidnapped recovery steps. 
Otherwise it will continue position tracking. 

The process starts with collecting odometry information, LRF scan and marker distance. If 
the LRF using map at a position according to the odometry differs significantly from the 
actual LRF scan properties (mean and standard deviations), it starts kidnapped recovery 
steps. Otherwise it continues position tracking steps. 

In the case of kidnapped recovery situation, the robot distributes particles [100 in this 
example] in the map randomly as shown in Fig. 15(b). Though in the actual map the 
area close to boundary is empty, the robot center cannot move very close to the wall or 
other obstacle because of the robot's size. This is considered in Fig. 15(b) where the 
robot will move only in the center area. The particles are distributed in this area to 
ensure efficient use of resources, because particles in the red area cannot represent the 
robot position and it is unnecessary to distribute particles in the area where the robot 
cannot go. 

In this work we considered only those LRF scan lines that would reflect from walls or 
obstacles and reflect from a range of 20 mm to 4095 mm. In this case the LRF scan data is 
considered reliable for determining the robot position in the map. After random distribution, 
the robot determines the ideal LRF scan mean and standard deviation at each location of the 
particles. It also determines the maker position according to the map with which the robot 
has been provided. Then it starts comparing the mean, standard deviation and marker 
distance according to the actual data and the data determined using the map. The closer the 
mean of a particle to the actual condition, the higher is the probability for the robot to be at 
that particle position in Fig. 15(c). 

If the robot fails to find a particle among the randomly distributed particles in one cycle it starts 
the whole cycle again until it finds a close position. During this time, instead of distributing all 
the particles randomly in the whole map, it uses the mean and standard deviations of the 
resultant particles at the end of each cycle to distribute particles for the next cycle. 
For position tracking steps, the robot knows its initial position. So when it moves, it 
distributes the particles in the direction it is moving with position information by 
odometry. Then, it determines the best particle to decide its position using LRF and 
marker information. 

(4) Result and discussion 

The robot determined its position in both position tracking and kidnapped condition 
within 100 mm of the actual position. In order to increase accuracy of the position, it will 
be necessary for the robot to repeat the same steps several times, which will take more 
time. This is a subject of ongoing research regarding map building and localization. If we 
make the map and set the target markers in the building for self-localization of robot 
manually, the robot can move in the building. So, the challenge is to make a more 
intelligent robot. 
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Fig. 14. Example of Map Building for Toshiba Science Museum. 
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(a) Localization error of ApriAttenda™ in the Toshiba Science Museum 
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(b) Random distribution of particles in the map for kidnapped recovery. 
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(c) Particle distribution after localization process. 

Fig. 15. Example of Self-localization in Toshiba Science Museum. 




Fig. 16. Scene of Experiment in Toshiba Science Museum. 
7. To Realize Life Support Robots 

The functionality of ApriAlpha™ as an information interface is upgraded so as to correspond 
to the needs in an environment where networked information appliances will be used. The 
person-following robot is intended to achieve coexistence with humans in terms of motion, 
not in terms of information. From now on, technology development will be continued with a 
view to performing actual tasks. 

7.1 Enhancement by information network technology 

To support everyday life, integration with an information network or sensor network is the key 
to offering services superior to those that an individual robot acting alone is capable of providing, 
as illustrated in Fig. 3 and discussed in Chapter 3. Such a robot is an active device in the network 
environment. Robots also need external sensors mounted in the environment. Thus, robots and 
information network need to collaborate with each other. In the near future, home-use robots are 
expected to be the core of home network systems. Also, home-use robots are expected to 
progress from those capable of simple functions such as cleaning to networked robots, and 
finally to multifunctional robots, as indicated in Fig. 17. Here, the same progression is considered 
not only for home-use robots but also for general service robots. 



7.2 Enhancement by environmental design 

Once robots enter widespread use in everyday life, it will be necessary not only to improve their 
ability, but also to improve their working environment. Information networking has become 
widespread. There is a need to define a common, friendly interface for both human and robots, 
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e.g. a barrier-free house. Here, we call it a universal design with robots (UDRob™) (Matsuhira et 
al., 2004), (Wada, 2004). Designing the environment both from the information side and the 
physical side, the realization of the life support robots will be facilitated as shown in Fig. 18. 
UDRob™ is based on the hypothesis that "an environment that is easy for anybody to use is also 
one in which it is easy for a robot to move around in and work." The improvement of the 
environment will greatly contribute to the performance of work by the robot, for example, by 
providing a physical environment where the robot can move easily with few disturbances, 
common mechanical interface for handling objects by gripping, and an information environment 
where the robot knows where it is and what is there. Although it is difficult to improve a robot's 
performance in terms of mobility, handling and visual recognition, these items are covered by 
environmental design, including the object interface. 

Mobility - if a wheelchair can move in the house a robot can also move. Handling - if the 
gripping interface which anyone can easily grasp is included in the concept of universal 
design, the robot can also grasp it easily. Vision - if a marker is designed clearly for a person 
who has poor eyesight, the robot can also recognize the marker easily. These are the concepts 
of UDRob™. Of course, RFID tags help provide information to the robot. Thanks to RFID 
tags, the robot knows the contents of a cabinet even if the robot cannot open it. But this 
method is unsuitable for humans because tags 7 information is invisible to human. So the 
UDRob™ concept includes tags with visible markers, such as simple icons of stationery, 
medicine, or tools, which are easily understood by both humans and robots. 
The robot's recognition ability by means of onboard computers is limited. The embedded 
knowledge in the environment via the network system helps the robot to execute tasks in 
daily life. For example, if an RFID tag has an object's properties such as weight, shape, 
length and knowledge of how the object should be handled by a robot, the robot can 
execute the task through the RFID tag. Many sensors are distributed in the environment 
to enhance robot performance through collaboration via information networking. This is 
why networked robots (Hagita et al., 2005) and ubiquitous robots (Kim et al., 2005) are 
currently under development. 
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Fig. 17. Development of home use robot as the core of the home network. 
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Fig. 18. Infrastructure improvement between information and physical environments. 

8. Conclusions 

The ongoing development of life support robots is presented by introducing the newly 
developed sharp ear robot ApriAlpha™ V3 and the person-following robot ApriAttenda™ from 
the viewpoint of human interfaces and mobile intelligence. In the future, by making full use of 
advanced network technology, home-use robots are expected to be at the core of home network 
systems and the widespread adoption of robots in everyday life is expected to be greatly 
facilitated by improvements in their working environment. Showing the concept of UDRob™, 
the environmental design including objects should be considered from the perspectives of both 
robots and humans. To realize life support robots, it is important to demonstrate what the robot 
can do in terms of actual tasks. The authors believe intelligent robots are the next technology 
whose development will decisively change the way people live. 

Other important issues are standardization of the robot's interface and safety problem. The 
activities of RSi (Robot Service Initiative) contribute to the common interface of information 
service such as weather forecast or news for service providers (Narita, 2005). In the OMG 
(Object Management Group) meeting, such an interface is also discussed (Kotoku, 2005), 
(Mizukawa, 2005). On the safety problem, the discussion on safety is an ongoing topic from 
the Aichi Expo. These activities will be fruitful in near future. 
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This paper constitutes the first part of a general overview of underwater robotics. The 
second part is titled: Underwater Robots Part II: existing solutions and open issues. 

1. Introduction 

The ocean covers about 71 % of the earth and each year of research and exploitation shows 
our ignorance and the difficulties we have in using this primordial and generous 
environment. Happily, this mysterious element has generated an unquenchable curiosity 
that pushed people like Le Prieur, Cousteau, Piccard, Walsh, and many others, to accomplish 
what was considered as impossible. 

B 




Fig. 1. Halley's diving bell, late 17th century. Weighted barrels of air replenished the bell's 
atmosphere. (U.S. Navy Diving Manual). 

In 1531, Guglielmo de Lorena dived on two of Caligula's sunken galleys using a diving bell 
on a design by Leonardo da Vinci. In 1776 David Brushnell invented the Turtle, the first 
submarine to attack a surface vessel, starting the long story of underwater warfare. In 
1933, Yves Le Prieur created the first constant-flow open-circuit breathing set, allowing 
for individual and light underwater voyages. In 1960, Jacques Piccard and Lt. Don Walsh 
touched the deepest point in the sea in the Mariana Trench (10916 m.) onboard the 
bathyscaphe Trieste, and confirmed that life is everywhere 1 . All these people 
demonstrated the feasibility of underwater intervention and circumvented the set of 
technical obstacles of such missions. Their first objective was to design system 
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guaranteeing the preservation of the operator's health. Removing humans from the 
immerged system reduces the critical contraints, but poses the problem of the 
guaranteeing the autonomy of the vehicle, and its effective capabilities. 
Navies were the first to show their interest in developing unmanned marine systems. In 
1866, the Austrian Navy asked Robert Whitehead to develop a new weapon for warship. He 
demonstrated the efficiency of a self-propelled floating device, carrying an offensive 
payload 2 . The torpedo vehicle class was born. The missions in which this vehicle was 
involved did not require any sophisticated control or navigation systems. This system was 
autonomous, as a bullet could be. 

In fact, the first Autonomous Underwater Vehicles (AUVs) were developed during the 60s 
and 70s with the SPURV (Self-Propelled Underwater Research Vehicle, USA) and the 
Epaulard (France). SPURV displaced 480 kg, and could operate at 2.2 m/s for 5.5 hours at 
depths to 3 km. The vehicle was acoustically controlled from the surface and could 
autonomously run at a constant pressure, sea saw between two depths, or climb and dive at 
up to 50 degrees. Researchers used the vehicle to make Conductivity and Temperature 
measurements along isobaric lines in support of internal wave modeling (Ewart, 1976). 
Epaulard was 3 tons and could operate by depth of 6000 meters during 7 hours, with a 
velocity of around 1 knot. An Ultra short Base Line (USBL) allowed for uploading orders 
and positioning relative to the mother ship (IOC UNESCO, 1985). 

These systems were the forerunners of the over 2400 unhabitated undersea vehicles, 
presently in regular operation worldwide (O.N.a. T.S., 1998 and Danson, 2002). 
The paper is organized as follows. The section 2 proposes to overview the current users that 
drive the development of underwater systems. Section 3 states the technological and 
theoretical demands in order to complete the desired autonomous operations. Section 4 
presents the notation that will be used throughout the paper. Section 5 introduces the 
current systems in use, detailing their properties from the robotics point of view. Section 6 
lists the desired performances in order to achieve the missions in a reliable way. Finally, 
Section 7 concludes this paper and Section 8 presents the references used in this paper. 

2. Forces in project development 

The rapidity in the development of robust systems routinely operated, or the design of 
new solutions, is strongly influenced by the investors involved. Designing an 
underwater system is an expensive task, but results in a cost-effective tool. In the 
commercial sector of the hydrographic industry, the UUV is already making a 
significant impact. Remotely Operated Vehicles (ROVs) and AUVs are currently 
routinely deployed for intervention on immerged structures or surveying wide range 
areas. However, private industry is not the largest sector that could, in the future, take 
advantage of these new technologies. Of the 1400 survey and research vessels in 
service world-wide, some 25% are engaged in missions related to public utility. Their 
work is principally focused towards charting navigation, for military and defence 
applications, for environmental assessments, monitoring, and for national economic 
application. Much of the world's marine data acquisition effort is concentrated within 
this range of activities; therefore much of the early development of AUVs was funded 
by the public sector (Danson, 2002). 
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2.1 The industry 

The biggest private users are the Gas and Oil companies that represent 58% of the offshore 
industry. 83% of its activity is taking place in water depths less than 300m, while the 
remainder is focused in the deep-water areas. As oil demand is increasing, this ratio will 
change with exploitation moving into ever-deeper waters (Whitcomb, 2000). Most of the 
currently used vehicles are Remotely Operated Vehicles (ROVs) designed to perform subsea 
inspection, construction, and repair operations. A ROV is teleoperated through an umbilical 
link, real-time connected to the operator, and has to be able to autonomously reach a desired 
location, search and lock onto a target on which the operator will perform manipulation: 
drilling, welding, configuring wellhead valve, plugging cables AUVs are involved in pipe 
inspection, terrain bathymetry and acoustic sediment analysis for pipeline installation and 
prospection for new oil and gas fields. 




Fig. 2. Underwater crawler vehicle being launched, National Institute of Ocean Technology, India. 



Non-living resource extraction is a major challenge for the next decade. For example, it is 
estimated that there are about 2,000 billion tons of manganese nodules on the floor of the 
Pacific Ocean near the Hawaiian Islands (Yuh, 2000), waiting for exploitation by Ocean 
Mining industries. The need is mostly similar to the oil and gas companies, except that the 
raw material dispersion requires the collector to move. This is a major problem and the 
actual technology does not yet provide a cost-effective solution that could allow for a 
generalized commercial exploitation. The present solutions propose heavy caterpillar- 
vehicles, teleoperated from the mother ship. This represents 10% of the global offshore 
industrial activity, and ocean mining is expected to be a growth area as the technologies are 
developed to harvest these products from the sea (Danson, 2002). 

Telecomunication is another sector of the offshore industry that is evolving rapidly. Its 16% 
share of activities is on the increase as the demand grows for secure, robust and high- 
capacity telecommunication network to link the nations of the world. Cable-laying and 
inspection requires AUVs with a specific control of payload-variation effects. ROVs are also 
used for cable repair, (Hartley & Butler, 1991). 

Other industrial sectors of the offshore industry are intended to grow rapidly in the near 
future: surface transportation (Caravella Consortium, 1999), fisheries assistance (The Ifremer 
Vital Campaign 3 ), support platform for divers... 
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2.2 The Navy 

Unmanned Underwater Vehicles are already in action in most of the military conflicts 
around the world. The 2004 US-Navy Unmanned Undersea Vehicle Master Plan (UUV 
Master Plan 4 , 2004) specifies the use of unmanned vehicles as force multipliers and risk 
reduction agents for the Navy of the future and postulates a host of specific missions for 
which UUVs are uniquely qualified. The long-term UUV vision is to have the capability to: 
1) deploy or retrieve devices, 2) gather, transmit, or act on all types of information, and 3) 
engage bottom, volume, surface, air or land targets (See Figure 3). 

According to the 2004 UUV Master plan, nine high-priority UUV missions have been 
defined, and, in priority order, are: 
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Fig. 3. the UUV Master Plan Vision, from (UUV Master Plan 4 , 2004). 



Intelligence, Surveillance, and Reconnaissance 

Mine Countermeasures 

Anti-Submarine Warfare 

Inspection / Identification 

Oceanography 

Communication / Navigation Network Nodes 

Payload Delivery 

Information Operations 

Time Critical Strike 

Recently, European Navies have opened invitation to tender on the subject of search and 
neutralize the remaining immerged explosive systems that are populating navigation routes 
and Atlantic coast since the 2 nd World War. This type of mission requires a global survey to 
locate and identity suspicious objects, using small AUVs equipped with Side Scan Sonar 
and/ or Electronically Pencil Beam Sonar. The problem of automatically identifying a 
possibly-dangerous-object in acoustic images is an open issue that is occupying a large 
number of robotic engineers. On a second level, object destruction is a dangerous and 
specific task that requires vehicles able to locate and approach the object to be neutralized. 
Atlas Electronik (Germany) and ECA (France) are proposing the SeaFox and Caster 
expandable vehicles, using teleoperation for the final approach and destructive phase. 



1 www.orionocean.org/PDFs/UUV_USNavy.pdf 
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2.3 Civil Protection 




Fig. 4. black-box recovery using ROV, form U.S. Dept. of Defence. 

The growth in world trade results in a massive increase in naval traffic. The Coast Guard 
intervention capacity has not followed economic expansion. The variety and unique nature 
of Coast Guard mission adds a high degree of complexity to technological solutions. 
Port Safety and security is a mission that involves many operational aspects. This implies 
an environmental responsibility for ports including shore side facilities. S.T. Tripp, in (Tripp, 
2001), addresses US Coast Guard UUVs needs during their specific missions. 

• Verifying that ships in port are seaworthy: ship hull inspection and internal inspection. 
ROVs are performing inspections that require a highly-precise relative positioning with 
respect to the hull. 

• Verifying that shore facilities are operating safely: ensuring that all channels are free of 
obstacle and correctly and adequately marked. AUVs are well suited for this kind of 
missions, with the condition that the navigation system addresses collision avoidance, in 
areas where there is high boat traffic. 

• Analysing and reacting to mystery sheen detection: mystery sheen is oil- or gas-like 
colouring in the water that has been spotted and reported to authorities. The source of 
these sheens typically is bilge pumped from a vessel in port. AUVs are tasked with 
sampling the sheen, with the condition that it is equipped with bio-chemical sensors able 
to surely identify the pollutant. The rapid reaction to the confirmation of the presence of 
a polluting slick could avoid dramatic oil or chemical tides. A fleet of Autonomous 
Surface Crafts (ASCs) are envisaged to be used, as described in (Jimenez et at., 2006), 
towing a boom in order to obtain a good confinement of the spill over. 

• Navigation aides: ensure the harbour entrance with autonomous surface scouts and 
determine the seasonal change in traffic. ASC are under development in many research 
labs, who are using these surface platforms to test and validate the algorithms that will 
be used on underwater systems. The problem of designing an autonomous tugboat 
implies being able to fuse many different aspect of the problem, including the control of 
traction effort. This is not an easy problem to solve. 

Law Enforcement concerns fishing restrictions and identification of polluters and 
pollutants. This involves detection, identification, tracking and interdiction of suspect 
vessels. AUVs coupled with various technologies to detect illegal activity and/ or 
sort/ classify targets of interest will provide the cost-effective, 24/7 platform to cover 
large restricted areas effectively. Recording a law violation implies to obtain a 
'prosecutable evidence 7 . Prosecutable evidence requires quality information of proven 
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accuracy, reliability, and authentication. Assuming the AUV could obtain prosecutable 
evidence, the system must communicate the certificated data to an interested party, 
which requires remote transmission of data such as video, high-definition still picture, 
recordings, and position data. Moreover, the AUV must get to a position to use these 
capabilities, which requires performing a series of complex maneuvers. These 
maneuvers include detecting a target, moving to a position where that AUV could 
gather the required information, providing an alert or direct data transmission, and 
perhaps relocating to a preferred download location. Because of the size of many 
restricted areas, the multiple AUV option would likely be the most practical option. 
The AUVs must be networked not only each other but also with other Coast Guard 
assets (planes, helicopters, ships and small boats). 




Fig. 5. The 'Prestige' tanker in distress, from CEDRE. 

A rapid Environmental Assessment provides deployed forces with environmental 
information in coastal waters in a specified time frame. The widespread observation and 
monitoring of ecological and environmental systems is of increasing interest in order to 
improve the management of scarce resources and also dramatically improve our ability 
to react to major pollution and natural cataclysms. The Ifremer institute is underlying the 
absolute need for an ocean monitoring network to keep watch over geophysical, 
biological and chemical phenomena, in order to detect the imminent occurrence of a 
geophysical cataclysm, like a tsunami 5 . The Autonomous Benthic Explorer (ABE) is an 
AUV designed to address the need for obtaining spatial coverage of a defined area, over 
a period of many months. ABE is intended to operate from a near bottom mooring from 
which it will detach itself periodically and fly or crawl pre-programmed surveys 
(Yoerger et ah, 2000). The data so obtained can be transmitted through an umbilical link 
connecting the surface to the moored docking bay, onto which the ABE will reattach and 
shut down for power conservation until the next survey. 

Arctic survey is considered as an issue of growing importance. The status of the 
circumpolar regions is a highly sensitive indicator of forerunner elements in global climate 
change. 2007-2008 is planned to be the third International Polar Year (IPY, the previous ones 
were held in 1882-83 and 1932-33), and will be an intense, internationally coordinated 
campaign of polar observations, research, and analysis that will further our understanding 
of physical and social processes in the polar regions, examine their globally-connected role 



3 http://www.ifremer.fr/com/actualites/tsunami.htm 
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in the climate system, and establish research infrastructure for the future (NSF, 2004). In this 
context, new underwater systems are currently in development, and will take place in the 
future Artie Observing Network. Autonomous Sub-ice survey allows for long mission time 
and range with no risk to human life. Nevertheless, a sub-icecap AUV navigation is a 
particular and complex situation, where the impossibility to reach the surface imposes a 
highly robust system, with restricted communication media. Numerous vehicles have been 
lost (and sometimes recovered) in such operating conditions. Climate history is studied on 
the icecap fringe, using ROVs, sampling ice at various depths. This application requires a 
highly precise relative positioning, with station keeping capabilities in an environment 
where external disturbances may be strong (Caccia et al., 2000). 

The previous items are of course not exhaustive, and we can list other civilian applications 
involving underwater systems: Nuclear Power plant pool and dam inspection and Black- 
box recovery. This last item is of major interest, since the increase in passenger 
transportation traffic will unfortunately be followed by a mounting number of crashes. This 
is a very complex mission that implies the consideration of many different situations. For a 
complete description of the problem, please refer to (Weiss et al., 2003). 

2.4 Scientific 

The oceans are still, for the most part, a mystery. For example, only for the year 2004, 106 new fish 
species have been discovered 6 . Scientific missions involving UUVS are numerous, and the duty of 
underwater robotics is to reply to the scientific demand in terms of data collection and acquisition. 
In the sequel, we list a series of successful robotics solutions in different fields. 
The Australian Institute of Marine Sciences is carrying out a survey project for Reef 
Management, designed to provide long-term quantitative data about corals, algae and 
marine life over the extent of the Great Barrier Reef. These data are for studies of abundance 
and population change in selected organisms on a large geographic scale. The reef 
surveillance task, as it is currently defined for the tethered Oberon vehicle, consists primarily 
of following an assigned path while maintaining a fixed altitude above the reef and 
avoiding collisions. Independent behaviours and arbiters, using decoupled controllers, have 
been developed as a modular means of accomplishing these various sub-tasks. For example, 
two behaviours have been developed for the path following aspect of the task; the first 
behaviour uses video input to track a rope laid out along the coral, while the second 
behaviour uses sonar to detect passive beacons. A centralized arbiter is then responsible for 
combining the behaviours 7 votes to generate controller commands (Rosenblatt et al, 2000). 
The protection of biodiversity requires a regular survey of well-chosen protected areas. 
Autonomous Marine Reserve Habitat Mapping allows for acquiring data with UUVs in 
order to estimate the health status of the protected zone. For example, marine biologists are 
watching closely the biological invasion of the algae Caulerpa Taxifolia, which is spreading 
over the Mediterranean Sea since 1984, from a common aquarium strain 7 . It is currently 
infesting tens of thousands of acres in the Mediterranean Sea and has now been found in 
two coastal water bodies in southern California 8 . Autonomously estimating the expansion of 
this parasite requires for the UUV to be able to extract from video images the border of this 



s http : / / www . delaplanete .org/ Renseignements-environnementaux,l 93 .html 

7 http://www.ifremer.fr/lerlr/caulerpa.htm 

3 http://swr.nmfs.noaa.gov/hcd/caulerpa/factsheet203.htm 
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seaweed field despite illumination problems, and follow this border in order to circumvent 
the colonized regions (ICES, 2006 and Rolfes & Rendas, 2004). 

Hydrothermal vent study interests a number of scientists. These Black Smokers are 
populated with specific ecosystems that may contain a large number of endemic species and 
provide constraints on the genetics and evolution of seafloor organisms. These particular 
systems are generally very deep and their analysis requires the use of ROVs or highly- 
manoeuvrable ABE to survey and sample the zone (Yoerger and Kelley, 2000). 
2005 was the international year of water. Many conferences and workshops have underlined 
the necessity for efficient and long term Water Resource Management. This implies a better 
management of existing resources, but also investigates the use of alternative water 
resources such as karst submarine springs. This is the aim of the European MEDITATE sixth 
framework programme 9 , for which the LIRMM will deploy the Taipan 2 AUV (Lapierre & 
Jouvencel, 2006) to carry out physico-chemical water sampling, in order to detect and 
authorize for public exploitation these underwater fresh water plumes. 
Another aspect of robotics application is the development of Biologically Inspired systems. 
This research is motivated by the intuition that natural solutions exhibit better performance 
than engineering designed systems. Indeed fishes are able to reach incredible records in 
terms of acceleration (250 m.s- 2 - northern pike fish - Esox Lucieus, Blake, 2004), velocity 
(46.3km/h, common eel - Anguilla Anguilla, Langdon, 2004 ; 108 km/h, Sailfish - Istiophorus 
platypterus, 10 ), turning radius (0.055 turning radius / body lengh, Knifefish - Xenomystus 
nigri, Domenici, 1997) and propulsion (520 horsepower, Blue Whale, Balaenoptera musculus, 
Joseph et al., 1988). Thus, many robotic research labs have developed prototypes inspired by 
tuna-fish (Pike Robot Tuna 11 ), eel-like robot (ROBEA Project 12 ) Manta ray (Manta Project 13 ) 
or even Turtle (Aqua Project 14 ). The first particularity of these systems is the lack of 
propulsion system. The thrust is generated by internal body-shape deformations. 
Controlling the movement of such systems implies fusing the requirements coming from the 
thrust generation and the control of the system trajectory. This considerable problem is still 
an open issue. Recently, propeller-free aeronautical-inspired systems have demonstrated 
high efficiency in terms of power consumption and range of action. These are the 
Underwater Gliders. The thrust is generated using an active buoyancy system (ballast) that 
creates a vertical force and makes the vehicle fly through the water column. A fleet of 
Slocum Gliders has recently been successfully deployed in the scope of the development of 
the New Jersey Shelf Observing System (Creed et al., 2002). 

3. The needs 

3.1 Hydrodynamic Modelling and Naval Architecture 

Designing an efficient underwater system requires a 'precise enough 7 knowledge of the 
underwater environment and the ability to model and predict the physical constraints 
the system will undergo during its mission. Borrowing from classic mechanics and 



9 http://www.meditate.hacettepe.edu.tr/partner/part_third.htm 
10 http:// www.elasmo-research.org/education/ topics/ r_haulin'_bass.htm 

11 http://web.mit.edu/towtank/www/Pike/pike.html 

12 http://www.irccyn.ec-nantes.fr/hebergement/ROBEA/ 

13 http://www.imasy.or.jp/~imae/kagaku/ 
14 http://www.aquarobot.net:8080/AQUA/ 
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approximated solutions of Navier-Stockes' equation, the modelling process allows for 
estimating the forces and torques applied on the vehicle, due to its interaction with the 
fluid and its actuation. This has of course a significant impact on the choice of the 
vehicle's shape and its actuation; a long-range system will be profiled according to the 
need to minimize the fluid effect in reaction of the system movement, while a system 
for pose-stabilisation will be designed to exhibit holonomic behaviour and isotropy in 
its capacity to react to external disturbances (Pierrot et al., 1998). Moreover, capturing 
in a significant manner the properties of movement of underactuated systems (case of 
the AUV) implies a dynamic modelling that injects nonlinearities and imposes to 
consider this model in the control design. A curious trade-off between precision of 
modelling and model controllability appears. The Society of Naval Architects and 
Marine Engineer is publishing a set of books of reference that are resuming the actual 
engineering knowledge in this field (Lewis, 1988). 

3.2 Positioning 

One of the most difficult questions to answer is the same since the origin of marine 
navigation, and concerns Positioning. This implies the ability for the system to locate 
itself on a geo-referenced map, and to estimate the relative position of the mission 
target. In air (aerial, terrestrial or surface), the geo-referenced positioning is facilitated 
by the use of GPS information. Underwater, this is no longer valid, satellites 
connection stopping at sea-surface. Different solutions are currently used. First, 
without GPS information, the classic dead-reckoning navigation is using 
proprioceptive sensors information and double-integrate acceleration meaurements to 
obtain an estimation of the displacement. This method is error-cumulative and implies 
a temporal drift in the precision of the estimation. Different strategies are used in 
order to upper-bound this drift. 

• The system can regularly reach the sea-surface in order to access to GPS information, 
and recalibrate its position. This solution is energy-consuming. 

• Buoys, immerged calibrated acoustic devices or even ASC systems can relay GPS 
information to the UUV, on condition that the horizontal relative distance geo- 
referenced-acoustic-device / UUV can be precisely estimated, (Pilbrow et al, 2002) , 
(Gomes et al, 2000) and (Vaganay et al, 1996). 

• A recent solution, named Simultaneous Localisation And Mapping (SLAM, Leonard et 
al, 2002) is demonstrating interesting performances. This terrain-based method uses 
visible (sensitive) land marks to build an egocentric map onto which all the a priori 
terrain-knowledge is projected. Then, revisiting a previously mapped region allows for 
associating the current sensor data with previously spotted land mark, thus bounding 
the position estimation error. 

• The relative position with respect to the mission target can be estimated using an 
appropriate sensor, without requiring a geo-referencing. This is called sensor-based 
navigation, and may require heavy data treatment such as feature extraction from image 
(video or acoustic). 

Moreover, data collected for later analysis can be meaningful if and only if, the precise 
location of the vehicle is known at the time the information is recorded. Then, a 
reliable UUV must be able to constantly determine its global position, and if possible, 
within certain certified error. 



318 Mobile Robots, Towards New Applications 

3.3 Motion Control 

As the previous chapter stated, the UUVs are tasked with different objectives, which require 
different strategies of Motion Control. 

• Point stabilizing: also called station-keeping, the goal is to stabilize the vehicle at a given 
point, with a given orientation, despite external disturbances. ROVs are well suited for 
this type of application, taking advantage of the holonomic and isotropic design that 
allows them to apply forces and torques on all the 6 degrees of freedom. Nevertheless, 
point stabilization presents a true theoretical challenge when the vehicle has 
nonholonomic (or non integrable) constraints, since there is no smooth (or even 
continuous) state feedback law that will do the job (Brockett, 1983). 

• Manipulating: the manipulation task implies many different aspects and requires 
decision capabilities that, for the moment, impose an efficient communication link 
with an operator. Assuming that the platform (the arm carrier) is able to maintain 
its pose - despite disturbances coming from sea current or umbilical and 
manipulator dynamic coupling effect - the teleoperation task requires the position 
control of the manipulator and the force control of the end-effector (tool). Recent 
applications using Intervention Autonomous Underwater Vehicle (IAUV), have 
demonstrated the feasibility of an autonomous underwater manipulation, 
controlled via an acoustic link (Badica et ah, 2004), thus removing the parasite 
effects of the umbilical cable. The underwater condition imposes a very low rate of 
communication (few bits/ second) and the teleoperation with delays is an opened 
issue (Fraisse et ah, 2003). 

• Trajectory-tracking: the vehicle is required to track a time-parameterized reference. The 
trajectory-tracking problem for fully actuated vehicle is now well understood. However, in the 
case of underactuated systems, that is when the vehicle has less actuator than state variables to 
be tracked, the problem is still a very active topic of research (Encarnacao & Pascoal, 2002). 

• Path-following: the vehicle is required to converge to and follow a path, without 
temporal specifications. The underlying assumption in path-following control is that the 
vehicle's forward speed tracks a desired speed profile, while the controller acts on the 
vehicle orientation to drive it to the path. Typically, smoother convergence to a path is 
achieved, when compared to trajectory tracking control laws, and the control signals are 
less likely pushed to saturation (Lapierre a & Soetanto, 2006) and (Lapierre b et ah, 2003). 

• Obstacle-avoidance: assuming that the UUV is equipped with proximity sensors able to 
detect the presence of an obstacle, obstacle detection can trigger many different 
reactions. This could be considered as a critical uncharted event and induces the system 
to stand by, waiting for operator orders. The imminent occurrence of a collision can 
trigger a mid-level reaction and proceed to a path replanning, or directly impact at the 
control level (Lapierre a et ah, 2006). 

3.4 Mission Control 

The previous section described various type of missions in which UUVs are deployed with 
different required degree of autonomy. But, even in a very simple mission contains different 
tasks to be sequentially executed, and the possibility to encounter uncharted critical 
situations has to be considered. The concatenation of these different tasks, or sub-objectives, 
and the reaction to uncharted events is in charge of the Mission Control system (Oliveira et 
ah, 1996). The transition between basic tasks is not a trivial problem, since each task may 
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require its own control law, navigation system, possibly interfering sensors, security 
manager... Then, each transition should be preceded with a global check of the system 
status before authorizing the transition to proceed. Moreover, the occurrence of an 
uncharted event has to produce a rapid system response. The mission control system is 
generally described using Petri Nets. Moreover, this tool allows for verifying system 
properties as determinism, and reactivity. The mission control aggregates low-level robotic 
tasks into mid-level behaviours, and organizes this collection of behaviour in order to 
complete the mission objectives, degrade them or even abort the mission according to the 
criticity of the uncharted events the system has met. 

Another important element, which could be considered as a part of the mission control 
system, is the Man/Machine Interface. This system is interfacing the machine with the 
operator, and is tasked with the mission programming - in terms that belong to the common 
end-user vocabulary - the mission monitoring - dependant of the communication media - 
and the results display and mission replay. 

3.5 Software Architecture 

Consisting of computers, micro-controllers, electronic interfaces and potentially interfering sensors, 
the system is conducted by the Software Architecture. This is the collection of the computer 
processes that has to be choreographed in order to realize, in a deterministic way, the set of robotic 
tasks that the system is nominally able to perform. The determinism implies to guarantee all the 
processes execution time and the coherence of the recruitment of the system's components. 
Acoustic emitters are of general use underwater and induce powerful insonification that might 
interfere with an inappropriate receiver. This problem is solved with a precise scheduling of the 
acoustic devices recruitment, cadenced by a deterministic and real-time task manager. 

3.6 Hardware Architecture 

The Hardware Architecture is the technological target onto which the previous software will 
be uploaded. This architecture has also to exhibit a deterministic behaviour, which impacts 
on technological choices. Most of the actual systems are over-dimensioned, ' statistically ' 
guaranteeing the system performances, but energy-consuming. Recent applications based 
on the co-design of the software and hardware architectures are currently working, with 
deterministic guarantee without requiring over-dimensioning. 



3.7 Control of Flotilla 




Fig. 6. The ASIMOV project, from ISR, Lisbon, Portugal. 
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Recently, there has been a surge interest in the problem of Coordinated Motion Control 
of Fleets of Autonomous Vehicles. The work reported in the literature addresses a large 
class of challenging problems that include, among others, leader-following, formation 
flying and control of the centre (or radius of dispersion) of swarms of vehicles. An 
interesting aspect of this problem concerns the control of flotilla composed by 
dynamically heterogeneous vehicles. Consider the application of the ASIMOV (Gomes et 
ah, 2000) Project where an ASC is required to follow a desired path, and an AUV 
operating at fixed depth is required to follow similar horizontal path (vertically shifted) 
while tracking the ASC motion along the original path. In this example the AUV acts as a 
mobile sensor suite to acquire scientific data, while the ASC plays the role of a fast 
communication relay between the AUV and the mother-ship and GPS satellites 7 net. The 
ASC effectively explores the fact that high data rate underwater communications can 
best be achieved if the emitter and receiver are aligned along the same vertical line. 
Notice that communication bandwidth is guaranteed if the vehicles are vertically 
aligned, within an upper-bounded limit defined by the emitting and receiving cones. 
This upper-bounded limit imposes a maximum horizontal relative positioning error, 
underneath which the communication is effective (Lapierre c et ah, 2003). The, the control 
objective is to guarantee this condition on the maximum relative position allowed. 

4. Notation and Definitions 
4.1 Notation 
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Fig. 7. Frames definition for an UUV. 
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Referring to Fig., the following notation will be used throughout the paper. 

The symbol {a} := {x A ,y A ,z A } denotes a reference frame with origin Oa. Let {17} and {B} 
be the inertial (or universal) and body axis frame, respectively. Traditionally, x,j is 
pointing north, yu is pointing east and z v is pointing the sea-bottom. The origin of {B} 

is usually chosen to coincide with the system's centre of gravity when this point belongs 
to the principal plane of symmetry, or at any other convenient point if this is not the 
case; x B is the longitudinal axis directed from aft to fore, y B is the transversal axis 
directed to starboard and z B is the normal axis directed from top to bottom. 
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Let R be the rotation matrix from the universal frame {B} to the body frame {IT}. 

Let i] = [x y z <fi 6 ^J^be the vector that expresses the body situation with respect 

to the universal frame. 

Let v = [u v w p q r] be the vector denoting the body's velocity with respect to 

the universal frame, expressed in the body frame axis {B}. 

Let r\ = k(i])- v denotes the system kinematics model, expressing the relations between 

T] and v . 

Let T be the vector of forces and torques generated by the actuation system, expressed 

in{B}. 

Let t = f(v,v,ti,P) denote the system dynamic model, where P is the vector of the 

constant physical parameters of the system, necessary to express the dynamic model. 

If x is a measurable physical quantity, let x be an estimation of x and x = x- x be the 

estimation error. 

Let t = B • u be the actuation model, where u denotes the vector of actuation inputs. 

4.2 Definitions 

The Navigation System provides estimates of the vehicle states based on a set of motion 

sensor suites. 

The Guidance System processes Navigation/ Inertial reference trajectory data and output 

set-points for desired vehicle's velocity and attitude. 

The Control System generates actuator signals to drive the actual velocity and attitude of 

the vehicle to the value commanded by the Guidance system. 

5. Systems 

Existing systems have been designed according to the required specificities imposed by the 
missions the systems are tasked to perform. From the robotic point of view, these characteristics 
concern the system actuation, the body shape, the communication medium and its bandwidth. . . 
and award the system with different properties and possibility of action. 



5.1 Remotely Operated Vehicle (ROV) 




Fig. 8. The ROV Panther Plus, from SeaEye Company, Fareham, England. 
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A ROV is generally designed to perform sub-sea intervention or survey of a delimited zone. 
These systems generally carry one or more manipulators, real-time-teleoperated via an 
umbilical link to the mother-ship. The umbilical and the manipulator generate coupling 
effects on the vehicle, beside the local environmental disturbances. Then, ROVs are 
generally exhibiting the following properties: 

i. Iso-actuated: the vehicle is equipped with as many actuators as the number of the 
controlled degrees of freedom. The remaining degrees of freedom (if any) have to be 
naturally stabilized, 
ii. Hovering: this imposes to the system being able to react in static water, which 
implies the use of propeller-, or jet-, driven thrusters. Moreover, this pose- 
stabilisation has to be guaranteed despite external disturbance, 
iii. Isotropic properties: the thrusters 7 location has to be chosen in order to guarantee 

that the system reaction capacity will be equivalent in all the direction, 
iv. Tethered: the umbilical is physically linking the immerged system with the mother- 
ship. This allows for on-line teleoperation and deported energy supply. Moreover, 
the presence of the tether improves greatly the guarantee of system recovery. 
Nevertheless, the cable is undergoing disturbances (e.g. waves and current) on all its 
length, inducing a heavy parasite-load on the system, 
v. Hybrid Position/Force controlled: performing a manipulation on an immerged 
structure implies to explicitly control the force exerted by the tool on the structure. A 
free-floating manipulation (in opposition to the clamped situation) requires 
combining the control of the vehicle's position, the manipulator effort on the 
structure and the compensation of the umbilical effects, 
vi. Terrain-based navigation: a precise estimation of the relative distance between the 
system and its target is of major importance in ROVs applications. First, it relaxes the 
precision necessity of the global positioning estimation, and second, it greatly helps 
in the vehicle stabilisation. 

5.2 Intervention Autonomous Underwater Vehicle (IAUV) 




Fig. 9. The IAUV Alive, from Ifremer and Cybernetix, Marseille, France. 
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The IAUV system is designed to perform similar operations than ROV, but without tether. 
The consequence is that the system carries its own energy supply. Moreover, manipulation 
is a complex task that involves a high degree of autonomy, which imposes for the moment 
to keep the operator in the decision and control loop. These systems are acoustically 
connected with the mother-ship. Their streamlined nature, in order to reduce the energy 
consumption and their ability to perform robust pose-stabilization require different types of 
actuation. These systems can combine control surfaces, stern powerful thrusters and 
hovering thrusters, awarding them with an over-actuated property. 

i. Over-actuated: the control of this type of system implies to solve an over- 
dimensionned problem, requiring the pose of complementary criteria, or to 
sequentially manage different actuation configurations. The stability property of the 
different switches between these configurations is a present subject of study, 
ii. Energy consumption management: this imposes to minimize the energy 
consumption, which constraints the hydrodynamic system shape and requires an 
explicit management of the sensors recruitment, 
iii. Transmission delays: the low bandwidth acoustic communication channel induces delays 

in the reception of the data. Teleoperation with delays is a very active topic of research, 
iv. Vertical acoustic transmission: since the best underwater communication is achieved 
when the emitter and the receiver are vertically aligned, the mother-ship has to be 
positioned on top of the IAUV horizontal location, guaranteeing that both systems remain 
in the communication cones, defined by the acoustic modem aperture. 

5.3 Autonomous Underwater Vehicle (AUV) 




» 



Fig. 10. The Taipan 2 AUV, from LIRMM, Montpellier, France. 

The AUV system is designed to perform long-range survey missions. Its fully-autonomous 
status implies an onboard decision capability and a rigorous energy consumption 
management. Moreover, the AUV generally carries powerful stern thrusters to propel its 
torpedo-shaped body in a preferred direction of movement 

i. Under-actuated: the vehicle has less actuator than states variable to be tracked. 

Control surfaces are used to induce a change in the relative fluid/ flow direction, 
ii. Actuated with control surface (fins): thrusters are loosing efficiency as the flow is 
misaligned with the propeller's axis. Since AUVs are running for high- velocity and 
long-range missions, the vertical plane actuation is performed using control surfaces. 
Notice that the system controllability imposes a non-null forward velocity; otherwise 
the control surfaces will not generate any significant action. Moreover, the control 
surface action capacity is dependant of the relative AUV/ fluid velocity, 
iii. Full-autonomy: The system decision capabilities will greatly impacts on the set of 
missions the system will be able to perform. The security management and the 
system ability to react to uncharted event are of major importance. 
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5.4 Autonomous Surface Craft (ASC) 

The ASC system is the surface version of the AUV. It benefits from a direct connection 
to GPS information. Nevertheless, surface sailing is sensitive to environment 
disturbances (wind, waves and current), and boat traffic requires an explicit 
consideration of collision avoidance. 




Fig. 11. The ASC Delfim, from ISR, Lisbon, Portugal. 



i. GPS navigation: the possibility to periodically estimate the global system 
coordinates greatly simplifies the navigation problem. 

ii. Collision avoidance: this implies the ability to detect an obstacle with radar (surface) 
and sonar (underwater) devices. The avoidance manoeuvre can be performed at 
different level in the control architecture: mission-replanning, path-replanning and 
low-level reflex behaviour. 

iii. Environmental disturbances sensitivity: wind and waves produce a forced- 
oscillatory disturbance on the system. Then, an ASC-based terrain survey 
(bathymetry, for instance), will have to be explicitly corrected in function of the 
attitude of the system, estimated by the navigation system. Indeed, the bathymetric 
measurements will be spread around the desired route, in function of the pitch and 
roll oscillatory behaviour. 



5.5 Glider Systems 




Fig. 12. The Slocum Glider, from Webb Research Corporation, Massachusetts, USA. 
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An underwater glider is a buoyancy-propelled, fixed-wing autonomous underwater vehicle. 
Attitude is controlled by means of internal mass redistribution and in some cases with external 
control surfaces. Initially conceived by Henry Stommel (Stommel, 1989), autonomous underwater 
gliders offer many advantages in ocean sensing: long duration missions, greater operational 
flexibility and low-cost operations. The thrust is achieved using transition between downwards 
and upwards glides, roll and pitch are controlled by moving internal mass. Yaw (heading) is 
controlled using the rudder mounted on the vertical tail of the glider. 

i. Active Buoyancy control: this allows for tuning the vertical position of the buoyancy 
centre, inducing an explicit control of the vertical force (heave), despite the velocity 
and direction of the flow. Notice that the system's vertical dynamics is dependant of 
the ballast-pumps flow rate, 
ii. Active mass control: the horizontal management of the weighting components of the 
system allows for tuning the horizontal position of the gravity centre. Combined 
with the buoyancy control, the system is able to generate a pitch control, 
iii. Hydrodynamic design: the shape of the projected system surface, perpendicularly to 
the fluid flow, has a significant incidence on the system dynamic reactions. As the 
weathercock aligns itself with the direction of the wind, the glider has to naturally 
pitch negatively (pointing down) when a descending force occurs (induces by 
negative buoyancy). This behaviour results in a system that is naturally orienting in 
the flow direction, inducing the desired flying effect. 

5.6 Biologically Inspired Systems 




Fig. 13. RoboEel mk2, from Modular Robotics and Robot Locomotion Lab, Singapore. 



Coming from the intuition that natural solutions exhibit good performances, 
biologically inspired solutions have motivated a large number of robotics applications. 
For example, Soryu I and II are three-tracked snake-like robot specially designed for 
rescue operation, to inspect the debris of building just after earthquake (Takayama & 
Hirose, 2003). Legged-robots are studied in many labs, and humanoid-systems are 
currently in development all over the world (Bryan et ah, 2000). For underwater 
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purpose, a simple look at the performances fishes can reach - in terms of acceleration, 
manoeuvrability and their ability to move by change of shapes, and not by propellers - 
justifies the marine engineers' interests in underwater biomimetic systems. Thuna- 
robot 11 , eel-like systems 12 and lobster robot 15 are currently under development, and 
theoretical opened problems are discussed in a large number of papers. 

i. Modelling: the modelling of a supple system, as the body of a fish, for control design 
purpose results in an hyper-redundant serial system. Classic dynamic modelling 
using Newton-Euler or Lagrangian-type derivation provides a complicated set of N- 
dimensionnal equations which requires specific mathematical tools to be solved, 
ii. Locomotion: the control of the movement of such a system borrows from vertebrate 
locomotion theory. In vivo observations of fishes and eels, allowed identifying periodic 
body-shape evolutions - also called gait - that induces an efficient thrust, 
iii. Gait control: the gait is classically a time-parameterized articular reference, 
tracked with a trajectory controller. The limitations of the trajectory tracking 
application have been exposed previously. Designing an autonomous gait (time- 
independent) allows for posing the control problem in terms of path following. 
Notice that the control loop is closed on an arbitrary reference, without direct 
measurement from the fluid flow. It results in a system that effectively produce a 
thrust, but without guaranteeing that all the system links (vertebrae) are 
properly involved in the movement, 
iv. Local flow control: fishes are using lateral line baro-sensors to locally control the 
flow quality along their body, in function of the locally measured pressure. An 
interesting approach is to reproduce such control ability in measuring the actuation 
torque in order to correct the nominal gait profile, and guarantee that the whole 
system is involved in the thrust production. 

5.7 UUV Platoons 

The problem of coordinated motion control of marine robots is a very active topic of 
research. The positioning problem can be greatly simplified with the use of an ASC, 
relaying GPS information to the immerged system. This implies to be able to precisely 
estimate the relative horizontal distance ASC/ UUV. Multiple UUVs control requires 
also estimating the relative position between the members of the platoon, in order to 
guarantee the formation cohesion. This implies a very-low bandwidth horizontal 
communication capacity that requires minimizing the necessary information exchange. 
The interest of making different vehicle collaborating is to sum the advantages and 
possibility of action of all. The consequence is that the platoon may be composed with 
systems that exhibit heterogeneous dynamical behaviours. 

i. Efficient Vertical Communication: the vertical plane induces the best 
communication rate. Then, an ASC/ UUV combined system is constrained by the 
maximum relative horizontal distance between the vehicles, as for the IAUV systems, 
ii. Minimal Horizontal Communication: The control of a UUVs flotilla requires 
estimating the relative position between all the platoon's elements. This estimation 
can be performed in a global way, with respect to a unique frame, or in a relative 
manner in estimating the distance to the closest vehicle. Nevertheless, exchanging 
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information in the horizontal plane is a difficult task and imposes to minimize the 
quantity of data exchange. 

iii. Coordinated manoeuvres: UUVs flotilla is composed with dynamical heterogeneous 
systems and implies different dynamics behaviour, in terms of minimum turning radius, 
reaction time. . . Moreover, in the scenario of a combined ASCs/UUVs control, the surface 
systems are undergoing different environmental disturbances than the immerged ones. 
Then, the guidance system of each vehicle has to consider manoeuvres generation 
according the system dynamics and the respect of the formation. 

iv. Obstacle Avoidance: operating different vehicle at a same depth requires an explicit 
consideration of the collision avoidance problem. Moreover, the onboard acoustic 
modem can be elegantly used to estimate and control the relative distance between 
vehicle and the respect of the formation. 

v. An efficient management of the terrain knowledge: in front of the difficulties 
of localising a deep UUV, terrain-based navigation offers an incontrovertible 
complementary solution. Simultaneous Localisation And Mapping (SLAM) 
allows for projecting on an egocentric map all the terrain knowledge, previously 
recorded or currently acquired, (Reece & Roberts, 2005). This map can help for 
navigation, when the sensors information can be currently associated with a 
particular location on this sensorial map. Coordinated mapping using a fleet of 
autonomous vehicle, enabling a global map construction is a challenging 
problem that raises a lot of exciting questions. Pooling the resources of each of 
the flotilla member into a collectively used global map is restricted with the 
communication underwater conditions. It is not conceivable that each vehicle 
broadcasts its locally acquired information, as done for aerial or terrestrial 
applications. Underwater, the transmitted data will be a degraded or reduced to 
a particular region of interest. It is also imaginable that the exchange of 
information could be done on demand, a vehicle requesting a complement of 
information in order to clarify an ambiguous measurement. The multi-SLAM 
principle is an appropriate tool to be integrated into the necessary collaborative 
space that UUVs fleet control requires. 

5.8 Navigation Sensors 

Underwater robotics would not be conceivable without the recent progress in the 
domain of sensing. Originally, sailors were using stars to triangulate their position, 
which is in fact not far from GPS technique. This archaic navigation was greatly 
improved with the invention of stable clocks (Eco, 1996). It is interesting to notice that 
accurate time measurement is still a requirement that is problematic to meet. As we will 
see later, precise synchronisation is one of the key factors in coordinating a fleet of 
collaborative vehicles. Moreover, time is a global reference that greatly simplifies the 
estimation of the relative position between a geo-referenced acoustic beacon and the 
UUV. Despite the time, the measurement needs, for navigation purpose, are related to 
position (and attitude), velocity and acceleration, as the structures of the vectors r\ and 
v suggest. Various sensors provide estimates of these quantities, with different 
accuracy and sampling rate. 

i. Magnetic compass is an old Chinese invention, which probably dates back to 100 BC, 
and is still in use everywhere. It is constantly pointing the north, hence allowing for 
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the estimation of the global heading ( y/ ) of the vehicle onto which it has been 
attached. Sensitive to the Earth magnetic effect, the magnetic compass may be 
parasited by the local magnetic field, generated by the active elements onboard the 
vehicle or present in the environment. 

This problem was overcome after the introduction of the gyroscopic compass. A 
gyroscope is a disk mounted on a gimbals in such a way that the disk can spin freely 
on it x- and y- axis. A properly mounted gyroscope will always turn to match its 
plane of rotation with that of the Earth, thus indicating the north along its axis of 
rotation without undergoing the effects of the local magnetic field. 
Fiber-optic gyro-compasses are based on the Sagnac 16 effect and offer great accuracy 
in the measurement of the yaw-rate ( f ) without requiring any moving parts. Three 
fiber-optic gyro-compasses mounted along the 3 axis of the body frame combined 
with 3 accelerometers, allow for estimating the complete set of the Euler angle 

( (/) , , y) ) without temporal drift. 

This sensors combination, called Inertia Navigation Unit, also provides estimations 
of linear accelerations ( U , v, w) and angular rates ( p , q , f ), besides ( <j) , 6 , y/ ). 
Moreover, given the initial geo-referenced location of the system, successive 
integrations of the linear accelerations provide the body frame velocities (u,v, w), 
and after appropriate transformation to the inertial frame, the global horizontal 
system position ( x, y). Notice that this integration induces a temporal drift in the 
estimation error. 

Doppler Velocity Loch (DVL) provides a great improvement, in directly measuring 
the body frame velocities ( U , V , W ) without requiring integration. Based on the 
Doppler effect, DVL is using the measurement of the difference in frequency between 
an emitted acoustic signal and the echo in return. In function of the insonification 
power and the temporal window of listening, the echo may come from the 
surrounding particle of water or the sea-bottom, thus providing the estimation of the 
body-frame velocities with respect to water, or inertial frame, respectively. This 
makes DVL used also as a currentmeter. 

Pressure sensors provide an estimation of the depth ( z ) without temporal drift. The 
previously described sensors compose a sufficient sensor suite to provide a complete 
estimation of the system states r\ and v . Nevertheless, the integration-based 
position estimation is error cumulative, and requires referring to an external 
calibrated reference in order to bound it. 

The most common approaches that UUV use are acoustic Long BaseLine (LBL), 
Short BaseLine (SBL) or Ultra-Short BaseLine (USBL) methods requiring 
external transponders. This is based on triangulation of the measurement of the 
time-flight of the acoustic signal between the vehicle and the geo-referenced 
transponders, effectively providing a non-drifting estimation of ( x , y , z ). 
However, signal attenuation varies with distance, frequency, and temperature, 
and positioning systems with acoustic beacons are expensive and often 
impractical (Vaganay et ah, 1996). Nevertheless, these transponders can be 
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located on ASCs that, thanks to their GPS connection, can regularly calibrate the 
beacons location. Moreover, a temporal synchronisation between the ASC and 
the UUV allows for estimating the UUV location with one single transponder, 
and ASC. The control of such a collaborative system involving underwater and 
surface system raises a lot of exiting problems. 

6. Sub-problems and Desired Performance 

The most displeasing experience with an UUV is to loose it, even for few minutes. This 
is an unacceptable eventuality and it is the goal of the system designer to reject its 
occurrence, while certifying the quality of the mission results in nominal conditions. 
Underwater, nominal conditions mean to define the domain of action of the system up 
to a certain sea-state, under which the system performances are guaranteed. These 
objectives are implicit guide-lines to the system design process. The analysis of its 
components, in order to evaluate these guarantees, reveals the necessity for all the 
components to exhibit global guaranteed performances. 

6.1 Modelling accuracy should be quantified. 

As we will see later this problem is complex, and theoretically unsolvable. The modelling 
objective is to cast the specificities of the underwater phenomena into a mathematical 
framework, which allows for predicting the system dynamic behaviour; in order to guide 
the control design and compute the actuation input that generates the desired movement. 
Rigid-body dynamics is now well understood, but viscous-fluid dynamics injects complex 
and nonlinear terms in the system model, which becomes difficult to control. The resulting 
system dynamic model is written as 

T = fi(v,V,fl,P) 

Environmental disturbance modelling results in a set of time-varying nonlinear 
equations. The consideration of their mathematical expression in the control design 
leads too much complexity. Hence, for control design purpose their effect on the system 
will be expressed as a 6x1 vector of torques and forces, w, acting on the system and the 
dynamic model becomes: 

T + W = f 2 (v,V,ll ; P) 

The assumption of a bounded effect of environmental disturbances, w, is related with the 
sea-state, and the presence of ocean current. Despite the environmental disturbances, 
modelling, model estimation and controllability-requirement impose approximations. With 
some notation abuse, the dynamic model is rewritten: 

T + W = f 2 (v / V,'n,PJ+f2 +f2(v,V,11,P) (1) 

where f 2 represents the unmodelled dynamics, P is theamic parameters vector and 
f 2 (v l v l ii l p) denotes the dynamic effects induced by the dynamic parameters misestimation. 
Then, the goal of the control design is to express a control expression based on the 
knowledge of f2^v,v,n,P] (which in the sequel will be written as f) in order to guarantee the 
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behaviour of the system described in Equation (1). Nevertheless, the assumption of bounded 
effect of the unmodelled dynamics is necessary, but unquantifiable in mathematical terms; 
since this phenomenon is excited by the system states, and the boundedness assumption 
implies to assume the system states stability; which can be guaranteed under the 
assumption of bounded external and unmodelled disturbances. This situation forbids a clear 
statement for the modelling error. Then, it is assumed in the sequel that the unmodelled 
dynamics f 2 is negligible. 

6.2 State estimation error has to be bounded 

State estimation concerns the estimation of the system position and velocities, assuming that 
these quantities are measurable, to provide rj and v . Nevertheless, the sampling periods 
and the precision of these measurements are generally very different, in function of the 
sensor or the measurement necessary conditions. Notice that the boundedness assumption 
implies for the system to be equipped with appropriate sensors. Global positioning requires 
the use of GPS information, directly acquired at the sea surface, or relayed by an acoustic 
device system. Without these measurements, a dead-reckoning navigation, based on the 
double-integration of the acceleration measurements, is error-cumulative, and the system 
states estimation error cannot be bounded. But a measurement of the global position, even at 
a very low frequency, will allow for bounding the estimation error. Of course, the goal is to 
obtain the best estimation, then to reduce as much as possible the guaranteed bound. The 
navigation system is in charge of this estimation. The objective is the following: given the 
sensors measurements denoted ^ , compute r\ and v such that r\ < z^ and v < £ v , where 

e^ and £ v define the maximum guaranteed error on the state estimate. 

6.3 Guidance and Control systems should exhibit global practical convergence 

Global practical convergence induces on the system to reach its objective, within a certain 
bounded error, whatever its initial position and the eventual obstacles the system has met. 
In the case of a pose-stabilisation this means that for any initial position \\q, the system is 

guaranteed to reach and stabilize on the desired position Hd , within a certain (hyper-) 

sphere Cl £ of radius 8 . For path-following, this property implies to reach the desired 3D 

path, within a certain (hyper-) tube B £ of radius G , whatever ijo • The respect of this 

condition impacts on the control design methods. 

6.4 Mission Control system and the Hardware and Software Architectures need to be 
reactive and determinist. 

Mission Control is tasked with the mission management. In a nominal situation, the 
mission is composed with sub-objectives, to be sequentially executed. These objectives are 
composed with elementary tasks that the system is nominally able to perform. The 
occurrence of an uncharted event, inducing a critical non-nominal situation, impacts on the 
mission planning (replanning or aborting). But in any situation, the Mission Control 
system has to deterministically guarantee that the system is constantly evolving toward a 
safe situation. The software architecture concerns the management of all the computer 
processes in charge of the elementary tasks of the system and has to exhibit real-time 
performances, in order to guarantee the execution time of all the processes. This is linked 
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with the choice of the operating system onboard the computer, the communication 
mechanism between processes, the tasks scheduling and the recruitment management of 
the system components. Moreover, since a system breakdown can never be completely 
dismissed, a self monitoring of the system is necessary. The hardware architecture has to 
also exhibit a deterministic behaviour, in order to guarantee that the previous properties 
will be respected by a reliable physical system. 

7. Conclusion 

The previously exposed requirements constitute a sufficient set of conditions in order to 
award the system with global guaranteed performances. These questions are currently 
subject of a great number of researches. We expose in the chapter titled Underwater Robots 
Part II: existing solutions and open issues, the state of advancement of these research, and the 
remaining open issues. 
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This paper constitutes the second part of a general overview of underwater robotics. The 
first part is titled: Underwater Robots Part I: current systems and problem pose. The works 
referenced as (Name*, year) have been already cited on the first part of the paper, and the 
details of these references can be found in the section 7 of the paper titled Underwater 
Robots Part I: current systems and problem pose. The mathematical notation used in this 
paper is defined in section 4 of the paper Underwater Robots Part I: current systems and 
problem pose. 

1. Introduction 

We propose in the sequel to introduce the existing solutions of the problems mentioned in 
the section 6 of the paper: Underwater Robots Part I: current systems and problem pose. We will 
present the solutions that allows for guaranteeing the desired global performances exposed 
previously. Section 2 presents the solutions to the Modelling problems. Section 3 introduces 
the different navigation systems currently in used. Section 4 proposes an overview of the 
classic Guidance strategies, and point out the specificities of this Guidance system that allow 
solving various problems. Section 5 presents the Control solution, underlying the ones that 
exhibit guaranteed performances. The Mission Control, Software and hardware 
architectures problems will be briefly mentioned in section 6. Finally section 7 concludes this 
paper and section 8 presents de references used. 

2. Naval architecture 

This development assumes the principle of superposition that is, for most marine control 
application, a good approximation. These modelled phenomena are described in 
(Newmann, 1977), and for the most significant ones to underwater robotics, listed below. 

2.1 Added Mass 

The hydrodynamic forces due to added mass are a consequence of the kinetic energy 
exchanges between the system and the surrounding water. Consider an object moving with 
a strictly positive acceleration with respect to the surrounding fluid. The energy expense of 
the system is caused by the system inertial forces, and the inertial forces of the surrounding 
particles of water that has to be compensated in order to induce the movement. Thus, added 
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mass has an inertial incidence, as the system mass itself. In a situation of a strictly negative 

acceleration with respect to the surrounding fluid, the moving water particles will restore 

their kinetic energy to the system. The consequence is that the global apparent mass of an 

immerged object is bigger than its intrinsic- (or dry-) mass. 

Notice that the added mass effect is a function of the fluid characteristics, the system 

geometry, and the body-frame-direction of the acceleration. Then, an immerged system will 

not have the same inertial behaviour - apparent mass - in all the directions, in opposition to 

the classic fundamental equation of the dynamics ( F = m • a ), where the mass is not a function 

if the direction of the acceleration. 

Moreover, the added mass forces are applied logically at the centre of gravity of the 

surrounding moving water. This point coincides with the system buoyancy centre, 

function of its geometry, while the dry-inertial effects impacts on the system centre of 

gravity. 

The contribution from this added mass phenomena are mathematically expressed as: 

x A =-M A v-C A (v)-v 

As for the rigid body dynamics, it is advantageous to separate the added mass forces and 
moments in terms which belong to the added mass system 6x6 inertia matrix Ma and the 6x6 
matrix of hydrodynamic Coriolis and centripetal terms denoted C A (v)- v • To derive the expression 
of these two matrices, an energy approach based on Kirchhoffs equations is applied. The reader 
will find in (Fossen, 2002) and (Newmann, 1977) the detailed background material in order to 
estimate the added mass coefficients of a given object shape. 

2.2 Hydrodynamic Damping 

D'Alambert's paradox states that no hydrodynamic forces act on a solid moving 
completely submerged with constant velocity in a non-viscous fluid. In a viscous-fluid, 
frictional forces are present such that the system is not conservative with respect to 
energy (Fossen, 2002). 

An object, immerged in a viscous fluid flow undergoes a force due to the relative 
velocity fluid/ object, which can be decomposed with a component along the velocity 
direction, the drag, and a perpendicular second component, the lift, as depicted in Fig. 1. 
The origin of these forces is the non-isotropic repartition of the pressure around the 
object. Notice that a spherical or cylindrical object will only undergo a drag effect, since 
this geometry induces a symmetrical flow separation whatever the relative flow 
direction. Hence, for ROV-type vehicle, where isotropy is a sought factor, the lift effect is 
generally neglected. In the case of AUV-type vehicle, the control surfaces are using the 
lift effect to create a change in the direction of the robot velocity with respect to the fluid 
flow. Nevertheless, the main body of the AUV generally exhibits planes of symmetry 
that justify neglecting the lift effect. Thus, lift effect is generally modelled and considered 
in the actuation model, in order to evaluate the control surfaces action. The drag and lift 
effect are modelled as follows: 

L =-•/?• C L .A(a)-|v t |-v t 
D =-•/?• C D .A(a)-|v t |-v t 
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Fig. 1. Lift and Drag effects. 

where L and D denote the lift and drag forces, respectively, p is the water density, a is 
the angle of attack, k(a) is the projected area perpendicularly to the flow direction and 
v t is the relative fluid / object velocity. C L and C D are the lift and drag coefficients, 
respectively. These coefficients depends of the Reynold number of the immerged object, 
which is a function of the object geometry, the fluid density and the relative velocity, 
(Goldstein, 1976). The drag effect, as previously described is in fact the most 
significant hydrodynamic damping undergone by a robotics underwater system, and is 
specifically called pressure drag. Other damping related phenomena exist. The skin 
friction is due to the water viscosity and the object surface roughness. It induces a 
water 'sticking 7 effect that engenders a small drag phenomenon. This is generally 
neglected in the scope of underwater robotics applications. The wave drift damping can 
be interpreted as added resistance for surface vessels advancing in waves. Notice that, 
as stated for the added mass, the point of application of these forces is the system 
buoyancy centre. The reader will find in (Faltinsen, 1995) and (Newmann, 1977) 
detailed material to estimate the Reynold number and drag and lift coefficients. 
Finally, the drag expressions acting on the 6 degrees of freedom of the solid are written as 
D(v)-v, where D(v) is the 6x6 system damping matrix. 

2.3 Restoring Forces 

Beside the mass and damping forces, underwater systems and floating vessels will also be 
affected by gravity and buoyancy forces. In hydrodynamic terminology, the gravitational 
and buoyancy forces are called restoring forces, and they are equivalent to the spring forces 
in a mass-spring-damper system (Fossen, 2002). The gravity force or system weight, W= pg , 
where m denotes the system mass and g the acceleration of gravity, acts at the system 
gravity centre. The buoyancy force is expressed as W = p • g • V where p is the water density 
and V is the volume of fluid displaced by the vehicle, and is applied at the volumetric 
centre (equivalently called buoyancy centre) of the displaced fluid. Notice that for a fully 
immerged solid the buoyancy centre coincides with the volumetric centre of the solid. Let 
r g=[ x g Yg z gF anci r b=[ x b % z bF define the- location of the gravity and buoyancy 
centres with respect to the origin of the body-frame {B}, respectively. Then, the 6x1 
vector g(h) expressing the restoring forces along the 6 degrees of freedom of the system 
can be written as in equation (2). 
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(2) 



In addition to the restoring forces G(ii), the system can be equipped with ballasts allowing for 
pumping water in order to modify the system weight, and as a consequence, the location of the 
gravity centre. Hence r g and W become controllable variables that allows for heave, pitch and roll 

dynamic stabilisation, for surface or semi-submersible vessels, depending on the number and the 
location of the ballast tanks (Faltinsen, 1995). Notice that underwater Glider systems are using this 
ballast control to alternate a the gravity / buoyancy dominancy in order to create a descending / 
ascending forces that induce a water flow in which the system flies. 

2.4 Environmental disturbances 

Environmental disturbances, due to waves, wind and ocean currents have a significant 
impact on marine system dynamics. Simple models for these disturbances applicable to 
control system design are found in (Fossen, 2002) and (Prado, 2004). A more general 
discussion on marine hydrodynamics is presented in (Faltinsen, 1995) and (Newmann, 
1977). Notice that wave and wind effects have an effective incidence on surface and sub- 
surface ASCs, and UUVs shallow water application. 

Winds affect surface vessels and their consideration is necessary for designing large crude 
carriers and tankers auto-pilots. ASCs are generally designed in such a way that wind 
resistance is negligible, but (Neal, 2006) is presenting an interesting study on a small-scaled 
sailing ASC and demonstrates the feasibility and interest of such a system. 
Waves are induced by many factors: wind, tide, storm surges and Stokes drift. A linear 
propagating wave theory is derived, borrowing from potential theory and Bernoulli's 
equations. It allows for a statistical description of wave based on an estimated wave 
spectrum. The wave elevation, <J Z , of a long-crested irregular sea propagating along the 
positive x-axis can be written as the sum of a large number of wave components, i.e.: 

n 

^ = X A i * sin ^i ' t-k i * x + ^j) 

Where Aj , co\ , k; and s\ are respectively the wave amplitude, circular frequency, wave 

number and random phase angle of wave component j. The random phase angles s\ are 

uniformly distributed between and 2n and constant with time. The wave amplitude Aj is 
distributed according to a wave spectrum estimated from wave measurements. 
Recommended sea spectra from ISCC (International Ship and Offshore Structures Congree) 
and ITTC (International Towing Tank Conference) are often used to calculate A ; (Faltinsen, 

1995). Underwater, co\ and k; are related with a dispersion relationship that rapidly 
attenuates the wave effects with depth. This attenuation effect is clearly stated in the 
expression of the horizontal fluid velocity <^ x and acceleration <^ x : 
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The wave-induced velocity and acceleration vectors of the surrounding particles of fluid are 
expressed as | and £ . 

The wave-induced oscillatory disturbance is rapidly attenuated with the depth, and in most 
UUVs application is neglected. But, ocean current is more problematic. Ocean current are 
horizontal and vertical circulation systems of ocean waters produced by tides, local wind, 
Stokes drift major ocean circulation, storm surges and strong density jumps in the upper 
ocean (combined action of surface heat exchange and salinity changes). As for the wave 
effect, ocean current induces on an immerged object a hydrodynamic load, due to the 
externally-induced movement of the surrounding particles of water. The low dynamic of the 
phenomena justifies assuming that the current is slowly-varying, hence the current velocity 
vector g , expressed in the universal frame {17} is considered as constant. 
Then the wave and ocean current induced disturbances are modelled as the universal velocity v w 
and acceleration v w of the surrounding fluid. Expressed in the body frame {B}, it comes: 

v w = R.'i+R.($ + g) 
Finally, considering the previously described environmental disturbances, the external 
disturbances acting on an immerged vehicle are expressed by the 6x1 vector w: 

w=M a (ti)-v w +C a (v w ).v w + D(v w ).v w 

Considering the superposition principle, this formalism allows for modelling the hydrodynamic 
forces that the system will undergo, in function of the system states r\ and v . A Newton-Euler 
analysis or a Lagrangian-type derivation allows for writing the equations of motion for an 
underwater robot without manipulator as follows (Fossen, 2002) and (Lapierre et al, 1998): 

i = K(n)-v 
M(i])-v + C(v)-v + D(v)-v + G(ii) = t + w (3) 

t = Bu 
where the kinematic relation is written as: 
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(4) 



M denotes the 6x6 symmetric inertia matrix as the sum of the diagonal rigid body inertia matrix 
and the hydrodynamic added mass matrix Ma, C(v) is the 6x6 Coriolis and centripetal matrix 
including rigid-body terms and terms Ca(v) due to added mass, D(v) is the 6x6 damping matrix 



340 Mobile Robots, Towards New Applications 

including terms due to drag forces, G(ti) is a 6x1 vector containing restoring terms formed by the 
vehicle buoyancy and gravitational terms and W is the 6x1 vector representing the environmental 
forces and moments (e.g. wave and current) acting on the vehicle. 

The actuation input u is composed with the thrusters' propeller angular velocity and the 
desired angle of the control surfaces. Thrusters' dynamics are nonlinear and quite complex. 
The reader will find in (Yuh, 2000), (Whitcomb and Yoerger, 1995), and the references 
therein, experimental elements to compare four thrusters models for blade-propeller type 
underwater thrusters driven by brushless DC motor. Control surfaces are using the lift effect 
to act on the system. Approximated theoretical solutions and experimental data have been 
collected to produce efficient models reported in (Fossen, 2002) and (Aucher, 1981). Notice 
that Equation (3) expresses a linear relation between x and u. This approximation is justified 
under the assumption that the thruster's dynamics have much smaller time constants than 
the vehicle dynamics [YUH]. Nevertheless, for control surfaces neglecting second order 
terms might be far from reality. Moreover, control surfaces are undergoing heavy 
hydrodynamic reaction forces, inducing deformations and a decrease in the drive motor 
response. 

The constant physical parameters of the actuation model compose the B matrix. Its components 
are dependent on each robot's configuration, control surfaces, ballast system, number and location 
of thrusters. Therefore B is generally not a diagonal, even square matrix. 

The environmental disturbance, w, induces a dynamic load on the system that can be 
considered during the control design, since many of these effects are suppressed in the 
closed loop. Moreover, ocean current induces on the system a static inertial-based load that 
modifies the working condition of thrusters and, as we will see later confronts an under- 
actuated system (ASC, AUV) with the Brocket's limitations (Brockett*, 1983) and impedes it 
to realize a pose stabilisation with a desired heading. 

When one or more manipulators are attached to the vehicle, it becomes a multi-body system 
and modelling becomes more complicated. The effects of the hydrodynamics on each link of 
the manipulators on vehicle motion have to be considered. Moreover, underwater 
manipulation requires the manipulator end-effector to exert a desired force on the structure 
on which the intervention is performed. Hence, the modelling has to explicitly consider the 
effects of the environment reaction on the system. Using the fact that intervention is done 
while the vehicle is station-keeping, the system model is simplified and decoupled 
(Whitcomb*, 2000), (Lapierre* et al, 1998) and (Fraisse et al, 2000). 

2.5 Biological Inspired system: theory of locomotion 

During a movement, fishes are continuously adjusting their body-shape in order to locally 
control the fluid flow along their body, hence reaching high efficiency in the thrust 
generation. This continuous deformation is difficult to model, and general modelling 
solutions consider a hyper-redundant N-serially-linked segments system. The purpose of 
the dynamic model is to express the actuation effects on the system states evolution. Then, 
the control analysis allows for designing an actuation pattern (also called gait) that 
"wiggles" the body surface in order to generate the desired thrust. The relationship between 
shape and position changes requires mathematical analysis borrowing from fluid potential 
theory and Lie algebra (Sfakiotakis & Tsakiris, 2004). Notice the use of Lagrange multiplier 
technique with tensor notation, to obtain a solution of the dynamic model of an eel-like 
system (Mclsaac & Ostrowski, 2002). 
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The physical constant parameters of the dynamic model (2) can be estimated using graphics 
in the literature. The dynamic parameters can also be precisely identified with 
hydrodynamic Tank Tests, reproducing an artificial fluid flow on the immerged structure, 
connected to a force sensor that allows for measuring the 6 forces and torques induced by 
the flow. Nevertheless, despite the precision of the model-parameters estimation, the 
modelling error has a significant impact on the control performances. Let P be the vector of 
the system dynamic parameters, P is an estimation of P and P = P - P denotes the modelling 
error. The control robustness denotes the system ability to react as desired, despite P . 
Moreover, the previous modelled phenomena expressions assume that the neglected high 
order terms induce minor effect. This might be empirically verified, but the theoretical 
consideration of the effects of the unmodelled dynamics is still an opened issue. This 
unmodelled dynamics has also another interest. Indeed, an accurate modelling that 
explicitly considers the high order terms will result in a precise description of the system 
dynamics, allowing for fine simulations. Nevertheless, modelling does not imply 
controllability, and as we will see in the sequel, the control design will impose to neglect 
some problematic coupled and non-linear terms of the model. 

3. Navigation System 

The Navigation System provides estimates of the vehicle states based on a set of motion sensor suites. 
As underwater systems have been developed, the complexity of the navigation problem 
increased. The demands are different in function of the system application. An ASC is riding 
the sea surface, and the navigation problem is to constantly estimate its global position with 
respect to the pre-defined geo-referenced route. The possibility to directly use GPS 
information and long-range radio link greatly simplifies the problem. A ROV navigation 
system will be tasked with estimating the necessary information in order to insure a robust 
and precise hovering control in front of the target on which the manipulation is performed. 
The approach phase requires similar information than for AUV, in order to control the 
system movement on a globally-defined or terrain-based route. Then, the navigation 
problem for UUVs is related with the estimation of the necessary information in order to 
accomplish the 3 following types of objectives: 

terrain-based pose-stabilisation , 

terrain-based route following, 

globally-defined route following. 
Underwater, the absence of direct GPS measurements leads different alternatives: 

periodic surfacing, 

pure dead-reckoning navigation, 

calibrated-acoustic-devices based navigation, 

terrain-based and SLAM navigation. 
In any case, multiple sensors are needed in order to provide a set of sufficient measurements 
to estimate r\ and v . UUV sensor suite is generally redundant, different sensors providing 
an estimation of the same physical quantity. Moreover, measurements related to the UUV 
movements are linked with the differential relation of the laws of mechanics. The navigation 
system is tasked with the fusion of these measurements in order to provide the best estimate 
of the system states. 

As mentioned in (Pascoal et al., 2000 and Oliveira, 2002) navigation system is traditionally done in 
a stochastic setting using Kalman-Bucy filtering theory (Brown & Hwang, 1992). The UUV situation 
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leads to consider nonlinear systems, and navigation solutions are usually sought by resorting to 
Extended Kalman filtering techniques. The stochastic setting requires a complete parameterization 
of process and observation noise, a task that may be difficult, costly, or not suited to the problem in 
hand. This issue is argued at length in (Brown & Hwang, 1972), where the author points out that 
in a great number of practical applications the filter design process is entirely dominated by 
constraints that are naturally imposed by the sensor bandwidth. In this case, a design method that 
explicitly addresses the problem of merging information provided by a given sensor suite over 
distinct, yet complementary frequency regions is warranted. 

Complementary filters have been developed to address this issue explicitly (Oliveira, 2002). 
In the case where the navigation sensors can be sampled at the same period, the 
corresponding filter operators are linear and time-invariant. This leads to a fruitful 
interpretation of the filters in the frequency domain, as briefly described in Fig. 2, in the 
situation of heading estimation. 
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Fig. 2. complementary filter estimating the yaw angle y/ based on the measurements of the 
yaw-rate y/ m and the yaw angle r m , from (Pascoal et ah, 2000). 
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Straightforward computation allows for writing the filter structure of Fig. 2 as: 

k M 1 

where the subscript Q m denotes the measurement, and \j/(s), y m ( s ) an( ^ i r m( s ) are me Laplace 

transform of y/ , y/ m and r m , respectively. Notice the following important properties: 

I T;l(s) is low pass, the filter relies on information provided by the compass at low 

frequency only, 
ii T 2 (s)= I -Ti(s). The filter blends the information provided by the compass in the low 

frequency region with that available form the rate gyro in the complementary region, 
iii The break frequency is simply determined by the choice of the parameter k . 
The frequency decomposition induced by the complementary filter structure holds the key of its 
practical success, since it mimics the natural frequency decomposition induced by the physical 
nature of the sensors themselves. Compasses provide reliable information at low frequency only, 
whereas rate gyros exhibit biases and drift phenomena in the same frequency region and therefore 
useful at higher frequencies. Complementary filter design is then reduced to the choice of k so as to 
meet a target break frequency that is entirely dictated by the physical characteristic of the sensors. 
From this point of view, this is in contrast with a stochastic approach that relies heavily on a 
correct description of process and measurement noise (Brown, 1972). 

In the case of linear position, based on low-rate acoustically relayed global measurements, 
and acceleration estimation based on some onboard accelerometers, the navigation system 
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has to merge inertial frame position with body-axis accelerations. This explicitly introduces 
the nonlinear rotation matrix R from inertial to body-axis in the filter structure. The 
resulting nonlinear filter is cast in the framework of Linear Parametrically time-Varying 
systems (LPVs). Using this set-up, filter performance and stability are studied in an H ^ 
setting by resorting to the theory of Linear Matrix Inequalities (LMIs, Boyd et al., 1994). 
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Fig. 3. complementary filter estimating the inertial position p and velocity v based on the 

measurements of the body-frame accelerations a m and the inertial position p w from (Pascoal 
et al, 2000). 

Fig. 3 describes a filter that complements position information with that available from 
onboard accelerometers, where a m =[ii m v m w m ] T denotes the measured linear 
accelerations along the 3 body-axis, p m = [x m y m z m f is the measured absolute position, 
v = [x y z] is the inertial estimated velocity and p is the estimated geo-referenced 

position. The structure design is then reduced to the choice of the gain matrices Ki, K2 and 
K3. (Pascoal et al., 2000) is converting the problem of filter design and analysis into that of 
determining the feasibility of the related set of Linear Matrix Inequalities. As a consequence, 
the stability of the resulting filters as well as their frequency-like performance can be 
assessed using numerical analysis tools that borrow from convex optimisation techniques 
(Boyd et al, 1994 and Brown 1972). 

Despite the time-varying problem, the characteristics of the sound channel imply that the 
position measurements are available at a rate that is lower than that of the velocity or 
acceleration sensors. To deal with this problem, (Oliveira, 2002) proposes an approach to 
navigation system design that relies on multi-rate Kalman filtering theory. Moreover, the 
author introduces some analysis tools to show that multi-rate filters can be viewed as input- 
output operators exhibiting // frequency-like ,/ properties that are natural generalization of 
those obtained for the single rate case. 

The filter of Fig. 3 proposes an interesting combination for the pose-stabilisation problem. 
Suppose a vision system providing an estimation of the relative position between a ROV 
and a visible landmark located on the structure onto which a manipulation has to be 
performed. The accelerometers and the vision system, equipped with appropriate feature 
extraction algorithms, provide precise necessary information to complete a hovering 
manoeuvre. (Kaminer et al, 2001) provides the detailed solution to this problem, applied to 
the problem of estimating the relative position and velocity of an autonomous aircraft with 
respect to a moving platform, such as a naval landing vessel. This solution allows fusing 
accelerometers measurements with information coming from the image space, using an 
adequate vision system. The consequence is the injection in the filter structure of the vision 
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sensor model, as depicted in Fig. 4. Fig. 4, where pj^ denotes the image coordinates of the 
desired feature, Ry am (t) is the rotation matrix from the universal frame to the camera frame, 
h^ is the calibrated vision-system model, expressing the relative position of the desired 
feature in the image frame p^ 11 from the relative feature position expressed in the camera 
frame and H T is the Jacobian matrix of h^ . 
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Fig. 4. complementary filter estimating the relative position of the desired feature with the 
UUV p 1 and velocity V based on the measurements of the body-frame accelerations a m 

and the position of the desired feature in the image plane, p'J^ 1 from (Kaminer et ah, 2001). 

Moreover, the authors of (Kaminer et al., 2001) show that the resulting filter exhibits 
guaranteed performances, that is: 

If I p llm -p'H < ^ p and | a-a m | < ^ a , 3e, such that p'= p'-p'< £ 

where Lis a bound on the measurement error, related with the sensor accuracy. The 

feasibility test, required for the choice of the gain matrices Ki, K2 and K3, can be iterated 
order to minimize the guaranteed bound £ , (Silvestre, 2000). An interesting unsolved 
question is related to the inverse problem: given a necessary £ , what should be the sensors 
characteristics, in terms of sampling frequency and accuracy, that provide an estimation 
guaranteed to be within £ . 

A generalisation of the previous results allows to state that, given a complete sensor suite 
that provides r\ m and v m , a combination of different complementary filters is able to 
provide an estimation of the system states such that: 

If I rf - Tf m I <(L and I v-v m | <^ v; 3(£ 77 ,£ 1/ j ; such that fj = t| — r| < c^ and v = v-v <£„, 

that is, the navigation system is exhibiting guaranteed performance. Notice that the condition 
I n - n m | < (I imposes the system to be equipped with appropriate sensors, including a device able 

to provide an estimation of the global position of the system, or at least able to counter-balance 
the systematic growth of the positioning error, induced by dead-reckoning navigation. As we 
have already suggested, a solution is to associate with the immerged system a calibrated 
reference device that is acoustically relaying the GPS information to the system. Being statically 
immerged, moored, drifting, or actuated, the use of these calibrated references greatly increases 
the operational and logistic burdens and its efficiency is dependant on the quality of the acoustic 
link. Terrain-based navigation offers a precious complement of information. Indeed a remarkable 
terrain feature can be used as a reference point in order to precisely estimate displacement. 
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Moreover, if this benchmark has been previously spotted and geo-referenced, then terrain 
navigation offers sufficient information in order to meet the guaranteed performance 
requirement, by exploiting the geometric and morphological characteristics of the environment; 
in particular sea bottom features such bathymetry, spatial distribution of biotypes distribution. 
This technique is named Simultaneous Localisation And Mapping (SLAM) and is currently an 
active topic of research (Leonard et al.*, 2002, Nettleton et al., 2000 and Rolfes & Rendas, 2001). 
The key point here is to guarantee the data association providing an unambiguous recognition of 
the spotted landmark. This implies to choose features that are robust with respect to the point of 
view (William et al. * 2002), which is a difficult problem. Indeed a wrong data association can 
drive the vehicle in a situation where it could be lost, justifying the necessity of combining both 
navigation strategies, using calibrated acoustic relays and terrain-based navigation. Notice that 
the SLAM aided navigation goal is not to produce a complete and globally consistent map of the 
environment, but only to provide sufficient environment knowledge in order to estimate, with 
the desired accuracy, the system displacement. Then the considered maps consist of a few 
number of relevant spatially features. In this way, the complexity of the filtering / estimation 
problem is limited and the map-data association is robustly solved (Rolfes & Rendas, 2001). The 
data-association is generally solved within a stochastic framework, estimating the result of the 
data association in confronting a representation of the posterior densities to the current sensors 
information, in order to predict the uncertainty of a given association attempt. Extended Kalman 
and Monte Carlo techniques are currently being used to solve this problem (Nieto et al, 2003). 
The particular problem of local coordinated navigation of an underwater vehicles flotilla can 
be elegantly posed using the multi-SLAM technique (Reece & Roberts*, 2005). As for a single 
vehicle, local coordinated navigation requires the estimation of the necessary information in 
order to guaranty the movement control of the group. As suggested before, ASC relaying 
geo-referenced information to the immerged vehicles is greatly helpful. Moreover, since the 
best communication rate is performed in the vertical water column, this vehicle can also be 
in charge of insuring the inter-vehicle communication. Nevertheless, the sea surface is not 
free of obstacle and it is expected that the ASC has to temporarily deviate form its nominal 
trajectory, thus compromising the communication net. Then, the immerged system has to 
share other kind of common references. SLAM technique offers interesting alternative to the 
centralized communication situation, and underlines the problem of extracting terrain- 
features that are robust with respect to the point of view. The sea bottom, at a given instant 
is a common reference for all the fleet members, and can greatly helps the relative 
localisation estimation. This implies that the vehicles are able to exchange information, 
which could be performed via the surface vessel. An alternative is to use a sub-surface 
system that can benefit from its vertical actuation in order to avoid obstacle without 
deviating form its nominal horizontal path. Moreover, this underwater system can adjust its 
immersion in order to optimize the acoustic cover on top of the flotilla. Nevertheless, 
despite the poor-rate horizontal communication capabilities, the possibility of a direct 
communication between two members cannot be neglected. Moreover, a precise temporal 
synchronization of all the fleet members allows for estimating the distance between the 
emitter and the receiver. That is precious information concerning the flotilla internal states. 
Moreover, in this context, an AUV can relay information to another one, which could be 
unreachable from the (sub-)surface vehicle. Inter-members communication implies to 
minimize the sum of the necessary information to exchange. In (Lapierre c et al. *, 2003) it has 
been shown that the collaboration between two underwater vehicles following the same 
horizontally shifted paths requires, besides the local navigation data, the mutual exchange 
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of the current control objective of both the vehicles. That is the current path-point each of 
them is currently tracking. The extension of this solution to N vehicles is still opened. The 
occurrence on an obstacle induces for the concerned vehicles to deviate from their respective 
paths. This may imply a reaction on the entire flotilla members, in order to keep the 
cohesion on the formation and insure a smooth return to a nominal situation. This necessary 
behaviour requires having regular information about the distance between the vehicles. A 
robust estimator of these distances has to fuse the (quasi-) global positioning estimation from 
the (sub-) surface vehicle with the estimation of the relative distance extracted from a 
measurement of the time-of-flight of the signal along the communication channel. Notice 
how a precise temporal synchronisation solves trivially the distance estimation. 
Complementary filters seem to be well suited to this problem, and the guaranteed 
performance they are offering is of major interest in this application. Fusing all the 
information related to the acoustical distance between each member towards all, with the 
one coming from the (sub-) surface vehicle, in considering the erratic sampling rates, is an 
exciting opened issue. 

4. Guidance 

The Guidance System processes Navigation/Inertial reference trajectory data and output set-points 
for desired vehicles velocity and attitude. 

In ROV systems, guidance commands are sent from a ground or mother-boat station, while 
AUVs have an onboard guidance processor. With regard to this, a guidance system plays 
the vital role in bringing autonomy to the system (Naeem et al., 2003). The guidance system 
computes the best approach to be followed by the vehicle, based on the target location, joy- 
stick operator inputs (if any), external inputs (weather data), Earth topological information, 
obstacle and collision avoidance data, and finally the state vector which is available as 
output from the navigation system (Fossen, 2002). Guidance system for underactuated 
marine vessels is usually used to generate a reference trajectory for time-varying trajectory- 
tracking or time-invariant manoeuvring for path-following. As a guide-line example, we 
have chosen to consider the guidance problem for path-following. The underlying 
assumption in path-following strategy is that the vehicle's forward speed tracks a desired 
speed profile, while the controller acts on the vehicle orientation to drive it to the path. 
Thus, as we will see in the sequel, the guidance problem for underactuated vehicles is 
reduced to the strategy in driving the desired heading angle of the system (y/d), while the 
desired forward velocity (uj) is left to the arbitrary choice of the mission designer. 

4.1 Set-point regulation 

A rudimentary guidance system, of general use for marine vessels, is called Set-point regulation. 
This is a special case where the desired velocity, position, and attitude are chosen to be constant. 
Applied to a surface craft, the desired attitude is reduced to the desired heading the vehicle has to 
follow. Then a desired pattern-following (in order to achieve a bottom acoustic coverage for 
example) will require from the guidance system a collection of n set-points defined by the three 
following characteristics: desired forward velocity, desired heading and duration, defining an 
element of the set-point database as: (u k y/ k t k ) T , for k=l,...,n. Notice that the environmental 
information is of major importance for this type of guidance system. Indeed the presence of a 
lateral current or wind will impact on the system trajectory without being compensated, resulting 
in a distorted achieved pattern. Nevertheless, this is the simplest guidance, and limited weather 



Underwater Robots Part II: Existing Solutions and Open Issues 347 

condition is enough to achieve simple missions as acoustic coverage of the seabed, since the 
acquired data will be post-processed. 

4.2 Way-point guidance 

Systems for Way-point Guidance are used both for ship and underwater vehicle. The system 
consists of a way-point generator with the human interface. The selected way-point are 
defined using Cartesian coordinated (x k y k z k ) T for /c=0,...,n, and stored in a way-point 
database. In the case where the path is only specified in the horizontal plane, only the two 
coordinates (x k y k ) T are used. Additionally, other way-point properties like speed (U k ), 
heading ( ^ k ) etc, can be defined. For surface vessels, this means that the ship should pass 
through way-point k at forward speed U k with heading angle y/^ . The heading is usually 

unspecified during cross-tracking, whereas it is more important during a crab-wise 
manoeuvre close to offshore installation, with the condition that the vehicle carries lateral 
thrusters in order to achieve the dynamic positioning. The way point database can be 
generated using many criteria, (Fossen, 2002): 

• Mission: the vessel should move from some starting point (x Yq z ) t to the terminal 
(x n Vn z n ) T , via the way-points (x k y k z k ) T . 

• Environmental data: information about wind, waves, current can be used for energy 
optimal routing (or avoidance of bad weather for surface vessels). 

• Geographical data: information about shallow waters, islands etc, should be included. 

• Obstacles: floating constructions and other obstacles must be avoided. 

• Collision Avoidance: avoiding moving vessels close to the defined route by introducing 
safety margins. 

• Feasibility: each way-point must be feasible, in that it must be possible to manoeuvre to 
the next way-point without exceeding maximum speed, turning rate etc. 

On-line replanning can be used to update the way-point database in case of time-varying 
conditions like changing weather, moving vessels, etc. In practice, the way-point guidance 
strategy can be cast in the set-point regulation description, in computing the current set- 
point characteristics in function of the currently tracked way-point k, and the system 
position, thanks to the following trivial computation: 

y/ k = arctan — (5) 

U-yJ 

and U k is left to the arbitrary choice of the operator. This solution constantly adjusts the 

desired vehicle heading toward the location of the way-point k. This techniques requires the 
definition of a threshold d wp , under which the vehicle is considered to be in an acceptable 
vicinity of the way-point k, and allows for iterating the process to the next waypoint k+1. 
Another option is to define a followable path, based on the way-point location. It is common 
to represent the desired path using straight lines and circles arc to connect the way-points 
(cf. Fig. 5 - dashed line). The drawback of this method, shared by the previous one, is that a 
jump in the desired yaw-rate n is experienced. This is due to the fact that the desired yaw- 
rate along the straight lines is r&J) while Yd =constant on the circle arc. Hence introducing a 
jump in the desired yaw-rate during transition from straight line to circle arc, or between 
consecutive way-points. This produces a small off-set during cross-tracking. 
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Cubic spline generated path results in a smoother reference and this drawback is overcome 
(cf. Fig. 5 - solid line). 
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Fig. 5. the path is generated using straight lines and circle arcs - dashed line - or using cubic 
spline interpolation - solid line. Line of Sight and Nonlinear Approach guidance problem pose. 

The path-following guidance strategy requires the definition of a more complex setting, 
introducing the problem of the path parameterization and the choice of the target-point to 
be tracked on the path. 



4.3 Line of Sight 

The Line Of Sight (LOS) strategy is an extension of the way-point guidance described in Equation 
(5), where the target point is not located onto the next way-point, but defined as the intersection 
between the path and the LOS vector of length h (horizon). The coordinates (x| 0S Yios) °^ * ne 
target point P/ os (cf . Fig. 5) can be easily computed in solving the following equations: 
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Notice that this is solvable if and only if h>yi, where yi is the cross-tracking error, that is the 
distance between the robot and the closest point on the path. The two previous equations 
have two solutions, and a contextual analysis allows for removing this ambiguity. 
A more general solution of the guidance problem, originally proposed in (Samson & Ait- 
Abderrahim, 1991), is the Nonlinear Approach Guidance strategy. It is based on the 
consideration of a Serret-Frenet frame {¥}, attached to the closest point on the path P. 
Consider Fig. 5, let y/ Q define the absolute angle of the tangent to the path at point P, and let 
6 = y/-y/ Q be the variable that the control system should reduce to as yi vanishes. This 
desired evolution of is captured in the definition of the approach angle £(y x ) : 



^(yi) = H9 a .tanh(k <r y 1 ) 



(6) 



where k^ is a positive gain and < <9 a <n\2 defines the asymptotic approach. That is when yi is 
big, the desired angular incidence to the path is 6> a , maximally defined as n/2 and inducing the 
desired approach to directly point toward P. As yi is reducing s(yi) decreases, down to when 
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the cross-tracking error is null. This method offers a smooth manoeuvre toward the path and, 
moreover, defines an appropriate framework in order to derive the control expression using of 
Lyapunov techniques. The global objective of the control is then to reduce the quantity 0-S(yi) to 
zero. Nevertheless, this method has a major drawback that consists in the consideration of the 
closest point P. Indeed, if the system is located the centre of the circle defined by the path- 
curvature present on the closest point, the guidance definition becomes singular and the y/ Q is no 
more uniquely defined. This singularity implies a restriction to the domain of validity of the 
derived control expression, inducing a highly-conservative initial condition y 1 (t = 0)<c c " 1 max / where 

c c max denotes the maximum curvature of the path. This restriction impedes the global nature of 

guaranteed performance requirement. Nevertheless, this solution stated a theoretical framework 
that allows for combined guidance / control design process, that will be used in the sequel. 



The risk zone. DVZ 




The virtual larpel 



Fig. 6. problem pose for the virtual target principle and the DVZ principle. 

4.4 Virtual target vehicle 

An alternative to this problem has been originally proposed in (Soetanto et al., 2003 and 
Lapierre b et ah, 2006) for an application to wheeled vehicle, extended to marine systems in 
(Lapierre a and Soetanto*, 2006). It consists in the development of the kinematic model of the 
vehicle in terms of a Serret-Frenet frame {F} that moves along the path; {F} plays the role of 
the body axis of a Virtual Target Vehicle that should be tracked by the 'real vehicle 7 . Using 
this set-up, the cross tracking distance yi and the angle 0-s(y 1 ) become the coordinates of 

the error space where the control problem is formulated and solved. A Frenet frame {F} that 
moves along the path to be followed is used with a significant difference with the previously 
described nonlinear approach guidance solution: the Frenet frame is not attached to the point on 
the path that is closest to the vehicle. Instead, the origin of {F} along the path is made to evolve 
according to a conveniently defined function of time, effectively yielding an extra controller 
design parameter s : where s denotes the curvilinear abscissa of the point P, on the path, 
thus capturing the evolution of the virtual target along the path (cf. Fig. 6). The consequence 
is the apparition of another variable Sj , and the control objective is to simultaneously 
reduce si, yi and 0-s(y 1 ) to zero. This seemingly simple procedure allows lifting the 
stringent initial condition constraint yi(t = 0)<c c _1 max , that arise with the path following 

controller described in (Samson & Ait-Abderrahim, 1991). The virtual target control law will 
be derived during the control design, allowing for a solution that exhibits global guaranteed 
performances. 
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4.5 Merging other requirements 

The previous solutions allow for guiding a marine vehicle toward a desired path. The 
problem of merging these guidance laws with other requirements, related to obstacle 
avoidance, environmental effects, optimal route planning, etc. is a difficult subject. Different 
strategies are proposed, in connection with the criticity of the requirement. For example, the 
occurrence of an obstacle must induce a rapid reaction, while information about a bad 
weather present on the current route will imply a path replanning without requiring a 
reflexive action. Nevertheless, the increasing computing power makes conceivable an 
obstacle avoidance based on path replanning, so far the obstacle is detected early-enough. 
The path replanning strategy requires the system to design a safe path between detected 
obstacles and excluded zones that warrants the vehicle to safely reach the desired goal. The 
proposed methods are generally classified in two categories: i) Graph Method and ii) 
Potential Methods. The graph methods are decomposed in two steps: i) the graph 
construction, allowing investigating all the possible paths and ii) the choice of the optimal 
path, in concurrently evaluating each of the solution performance (Latombe, 1991). The 
potential method is based on the space decomposition in terms of fictive potentials. An 
obstacle or an excluded region will induce a repulsive potential while an attractive potential 
will be placed onto the goal location. Then searching for the lines of minimum potential 
allows for planning the possible paths. Analyse of their performances in terms of energy 
consumption or risks, allow extracting the optimal solution (Barraquand et al, 1992). 
Nevertheless, these methods do not guarantee the absence of local minima, and the selected 
path may drive the robot at an undesired impasse (Koditschek 1987). In (Louste & Liegeois, 
1998), the authors propose a method based on the Navier-Stokes equations of incompressible 
fluid, undergoing a difference of pressure between the origin and the goal. As the water will 
always find a way to leak (if possible) form a pressurized confined environment, the 
analysis of a simulated flow, extracting the routes of maximum fluid particles velocity, 
allows for finding a global route that travels between obstacles and reach the goal without 
encountering local minima. Under the assumption that an accurate map of the environment 
is available; it designs a global solution that guarantees the system to reach the goal, if this 
solution exists. The problem of coupling this solution with the kinematics requirements of 
the vehicle is still unsolved. Nevertheless, the simulation of the fluid flow induces heavy 
computational burden that disqualify this solution in a real time context. 

4.6 Reactive obstacle avoidance 

A reactive obstacle avoidance strategy will be preferred. This solution is using the 
Deformable Virtual zone (DVZ, Zapata et al, 2004) principle where a kinematic-dependant 
potential is attached onto the system, in opposition to the classic potential methods where 
the map is potentially-active. The idea is to define around the robot a risk zone, the ZVD, in 
which any obstacle penetration induces a repulsive reaction. The shape of the ZVD is based 
on an arbitrary choice (generally elliptic), whose parameters are governed by the vehicle 
kinematics and state evolution. Since the ZVD is attached to the vehicle, both are sharing the 
same non-holonomic, or underactuated, constraints. The obstacle intrusion is quantified as 
the area of the penetration (I), cf . Fig. 6. Tedious but straightforward computation yields the 
jacobian relation between the vehicle velocities (u and r, for unicycle-type robot, v for 
marine vehicle) and I. The control design framework is then well posed, and a combined 
(guidance / obstacle-avoidance / control) design process is possible, in order to seek for 
solutions that exhibits global guaranteed performances, as detailed in (Lapierre a et al.*, 2006). 
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The virtual target principle needs to be deeper investigated. An interesting extension is to attribute 
to the virtual target another extra degree of freedom, y s . This could allow the point P to leave the 
path laterally, and design a virtual target control in order to fuse the all the requirements on this 
runner. Moreover, an adjustment of the si variable will allow for using a second virtual target as a 
scout in order to provide a prediction, compatible with the control theoretical framework. 

4.7 Deformable constellation 

The consideration of the guidance problem in a multi-vehicles context is an exciting question, where 
the presence of obstacle in the immediate vicinity of the vehicles is omnipresent. A vehicle deviating 
from its nominal path may imply a reaction on the entire flotilla members, in order to keep the 
cohesion on the formation and insure a smooth return to a nominal situation. The principle of the 
Deformable Constellation, introduced in (Jouvencel et al, 2001), allows for fusing different criteria, 
related to communication, minimal distance keeping and mission objectives (optimizing the 
acoustic coverage of the seabed, for example), and attribute to each member the appropriate 
individual guidance and control instructions. The theoretical framework of this solution needs to be 
clarified in order to extend its application and evaluate the guaranteed performances of this 
solution. Based on an extension of the Virtually Deformable zone, this solution allows conceiving 
the creation of an effective collaborative space, for which the objective of the navigation systems of 
all the members is to complete the knowledge. In this scope, the constellation guidance is not any 
more defined around an arbitrary formation, but governed by the obligation of particular 
measurements, prioritized in function of their necessity. This guidance problem of a flotilla in order 
to optimize the collaborative acquisition of a desired measurement is a hot topic of research. 

5. Control 

The Control System generates actuator signals to drive the actual velocity and attitude of the vehicle 
to the value commanded by the Guidance system. 

The control problem is different in function of the system actuation and the type of mission the 
robot is tasked with. The actuation effects have been considered during the modelling process. 
While the Navigation system is providing an estimation of the necessary variables, the goal of the 
guidance system is to take into account the system holonomic property and the type of missions 
(pose stabilisation / long range routing), in order to cast the control problem under the form of 
desired values ti^ and v^ to be tracked by r\ and v , thanks to the control system. 

5.1 Hovering 




Fig. 7. The URIS ROV, Univerity of Girona, Spain. 
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The operating conditions allow for hydrodynamic model simplifications, and a pose-stabilisation 
problem implies small velocities that greatly reduce the model complexity. Moreover, vehicles 
designed for hovering are generally iso-actuated, or fully-actuated in the horizontal-plane and in 
heave (immersion), while the roll and pitch dynamics are passively stable (see, for instance the 
URIS ROV, Fig. 7). Then, hovering controller for ROVs are generally based on a linearization of 
the model of Equation (3), resulting in conventional PD or PID control laws (Whitcomb*, 2000). 
The navigation system, coupled with vision or acoustic devices provide a precise estimation of the 
vehicle pose, using for example the complentary filter depicted in Fig. 4 that fuses acceleration 
measurements with the vision system data, or the solution proposed in (Perrier, 2005) fusing Loch 
Doppler system velocities with dynamics features extracted form the video images. 
Pose-stabilisation is an adequate situation to meet the linearizing condition requirements: i) 
small roll (</>) and pitch (6> ) angles, ii) neutrally buoyant vehicle (W = B and r g = r b ) and iii) 

small velocities (v). Considering these approximations and expressing the system model, 
Equation (3) in the Vessel Parallel Coordinate System {P} (a coordinate system fixed to the 
vessel with axes parallel to the Earth-fixed frame) allows for writing the system as the 
disturbed Mass- Spring-Damper system expressed in Equation (7). 

M -v + D-v + K -iip =T Pm +w (7) 

Where M, D and K are constant matrices and t| P is r\ expressed in {P}. Classic methods for loop 

shaping allows for computing the appropriate values of the classic PID gains that results in the 
controlled forces and torques x P | D . Nevertheless, a classic PD controller is reacting to the detection 

of a positioning error, and as a consequence, exhibits poor reactivity. The adjunction of the integral 
term, resulting in a PID controller, is improving this situation in implicitly considering a slow- 
varying external disturbance. Nevertheless, the low-dynamics integral action cannot provide the 
desired robust-stabilisation in a highly-disturbed environment. An interesting solution, called 
Acceleration Feedback, proposes to add an external control of the acceleration, t af = K AF • v , in order 
to consider 'as soon as possible 7 the occurrence of a disturbing action w on the system, where K AF 
is a positive diagonal gain matrix, resulting in the following closed loop expression. 

M -V+D-V+K Tip = Tp| D +x AF +w 

Equivalently, and with some notation abuse, 
D K 



-lip 



M+K AF M+K AF r M+K AF riu M+K AF 

From this expression, it is noticed that besides increasing the mass from M to M + K AF , 
acceleration feedback also reduces the gain in front of the disturbance w from 1/M to 
1/(M + K AF ) . Hence, the system is expected to be less sensitive to an external disturbance 
w if acceleration feedback is applied. This design can be further improved by introducing 
a frequency dependant acceleration feedback gain Taf =H AF (s)v, tuned according to the 
application. For instance, a low-pass filter gain will reduce the effects of high frequency 
disturbance components, while a notch structure can be used to remove l st -order wave- 
induced disturbances (Sagatun etal, 2001 and Fossen, 2002). Nevertheless, accelerometers 
are highly sensitive devices, which provide a high-rate measurement of the accelerations 
that the system is undergoing. As a consequence these raw measurements are noisy, and 
the acceleration feedback loop is efficient in the presence of important external 
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disturbances, guaranteeing the significance of the acceleration estimation, despite the 
measurement noise. 



5.2 Manipulation 

Recall that a precise dynamic positioning is of major importance for hovering control, especially if 
a manipulation has to be performed. Then, the manipulator and umbilical effects have to be 
explicitly considered. Moreover, as the simple presence of an umbilical link induces a dynamic 
effect on the vehicle, the manipulator that moves in a free space, without being in contact with a 
static immerged structure, generates also a coupling effect. This coupling effect is due to the 
hydrodynamic forces that react to the arm movement. A first approach is to consider the complete 
system (vehicle + manipulator), resulting in an hyper-redundant model expressing the dynamics 
of the end-effector in function of the actuation. Despite the linearization simplifications, the model 
remains complex and the control design is difficult and the performances are highly related to the 
accuracy of the model identification. Computed torque technique, (Gonzalez, 2004), allows for 
estimating the coupling effect on the link between the vehicle and the manipulator. Then the pose- 
stabilisation problem of the platform and the generation of the manipulator movement control are 
decoupled. Same approach can be used in order to compensate for the umbilical effect, meaning 
that a precise model of the hydrodynamical forces undergone by the cable is available. This is a 
difficult task since the umbilical cable is subject to disturbances along its entire length and the 
modelling requires having a precise knowledge of the currents and wave characteristics. An 
alternative, exposed in (Lapierre, 1999), proposes to use a force sensor placed on the link between 
the manipulator and the platform, in order to have a permanent measurement of the coupling. 
This coupling measurement, denoted F ufyman, [ s used to feed an external force control loop that 
corrects the position control of the vehicle (cf. Fig. 8). 
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Fig. 8. problem pose and hybrid Position/ Force external control structure. 

Notice that the use of a single force control loop results in a reactive 'blind 7 system that 
exhibits a position steady-state error, while a single position control loop slowly, but 
precisely, correct the position error. Hence, the simultaneous control of the platform 
position and the coupling effect combines both the advantages of the force control reactivity 
and the precise steady control of the position. The manipulation generally consists in 
applying a desired force on an immerged structure on which an appropriate tool is 
performing the operation (drilling...). In this case, the coupling forces and torques present 
on the link between the manipulator and the platform is also due to the environment 
reaction to the operation. A steady state analysis underlines the necessity for the platform to 
apply the end-effector desired force on the coupling articulation. Nevertheless, since the 
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system is in contact with the environment, the coupling dynamics depends on the 
environment characteristics, generally modelled as a mass-spring-damper, and the 
thrusters' dynamics mounted on the vehicle. The solution proposed in (Lapierre, 1999) 
consists in a gain adaptation of the platform and of the manipulator controllers in order to 
combine the dynamics of both subsystems. Then, the low response of the platform is 
compensated by the high reactivity of the manipulator. This allows for performing free- 
floating moving manipulation, as required, for instance, for structure-cleaning applications. 
Recent experimentations on the ALIVE vehicle 1 have demonstrated the feasibility of a simple 
underwater manipulation via an acoustic link, removing the umbilical cable necessity, and its 
drawbacks. The poor-rate acoustic communication does not allow real-time teleoperation, since 
real-time images transmission is impossible. Then, the teleoperation loop has to explicitly consider 
varying delays that greatly complicate the problem. A solution to this problem is detailed in 
(Fraisse et al.*, 2003), and basically proposes to slow-down the manipulator time-response, in order 
to adapt the delicate force application to the erratic incoming of the reference, provided by the 
operator. The target approach phase requires the Intervention AUV (IAUV) to navigate over a 
relatively long distance, and it has to exhibit the quality of an AUV system. Indeed, the inefficiency 
of side thrusters during a high-velocity forward movement leads to consider the IAUV system as 
underactuated. Notice that a controller designed for path-following cannot naturally deal with 
station keeping, for underactuated system. This limitation has been clearly stated in (Brocket*, 
1983), and can be intuitively understood as the impossibility for a nonholonomic system to 
uniformly reduce the distance to a desired location, without requiring a manoeuvre that will 
temporarily drives the vehicle away from the target. Moreover, in presence of ocean current, the 
uncontrolled sway dynamics (case of the underactuated system) impedes the pose-stabilisation 
with a desired heading angle. Indeed, the single solution is for the underactuated vehicle to face 
the current. As a consequence, IAUV systems are fully-actuated, but can efficiently manage the 
actuation at low velocity. The first solution consists in designing two controllers and switching 
between them when a transition between path-following and station keeping occurs. The stability 
of the transition and of both controllers can be warranted by relying on switching system theory 
(Hespanha et al, 1999). The second solution consists in designing the path-following algorithm in 
such a way that it continuously degenerates in a point-stabilisation algorithm, smoothly adding 
the control of the side-thrusters, as the forward velocity is decreasing, retrieving the holonomic 
characteristic of the system (Labbe et al, 2004). Notice that the powerful stern thrusters are not 
suited for fine control of the displacement. Then, these vehicles are equipped with added fine 
dynamic-positioning thrusters that lead to consider the system as over-actuated during the 
transition phase. The control of this transition implies to consider sequentially an uderactuated 
system, an over-actuated system, and finally an iso-actuated system. This specificity in the control 
of an IAUV system is a current topic of research. 

5.3 Long-range routing 

Control design for underactuated marine vehicles (AUVs, ASCs) has been an active field of 
research since the first autopilot was constructed by E. Sperry in 1911 (Fossen, 2002). Basically, it 
was designed to be a help for ship pilots in the heading control, while the forward movement was 
tuned according to a reasonable motor regime. Providing an accurate yaw angle measurement, 
classic PID controller allows for driving any conventional ship to a predefined list of set points. 



1 http://www.ifremer.fr/flotte/coop_europeenne/essais.htm and cf. Figure 9 in the paper Underwater 
Robots Part I : current systems and problem pose. 
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Enriching the navigation system with GPS measurements extends the application of this strategy 
to way-point routing and LOS guidance technique. Nevertheless, this seemingly-simple control 
scheme hides a complex problem in the gain tuning, for who requires the system to exhibit 
guaranteed performances, that is bounding the cross-tracking error along the entire route. 
Linear Quadratic technique allows for designing a controller for the linearized system, 
which minimizes a performance index based on the error and time-response specifications 
(Naeem et al., 2003 and Brian et al., 1989). The linearization process of the model of a vessel 
in cruising condition assumes, upon the relevant conditions previously listed in the station- 
keeping case, i) a constant forward velocity (11=11^ ) and ii) a small turning rate (t| p « v ). 
This results in the state-space linear time invariant model: 

x = A-x+B-u+E-w+F-v n 

(8) 
y = Cx K) 



where 



T~l 

^p 1 (v-v ) , v o=[ u d 0] T ,u=r. The expression of the 12x12 



matrix A , the 6x12 matrix C and the 12x6 matrices B, E and F can be found in (Fossen, 2002). 
The control objective is to design a linear quadratic optimal controller that tracks, over a 
horizon T, the desired output yd while minimizing: 
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J =min -• he 1 Qe+u 1 -R-uj-dtf 
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where Q and R are tracking error and control positive weighting matrices. It can be shown 
(Brian et al., 1989) that the optimal control law is 

u = -R" 1 -B T -[P-x+h 1 + h 2 ] 

where P is a solution of the Differential Riccati Equation, and hi and I12 originates from the 
system Hamiltonian, and can be computed according to (Brian et al, 1989). 
Another approach, called Feedback Linearization, proposes to algebraically transform a 
nonlinear system dynamics into a (fully or partly) linear one, so that linear control 
techniques can be applied. This differs form conventional linearization, as exposed before, in 
that feedback linearization is achieved by exact state transformations and feedback, rather 
than by linear approximations of the dynamics (Slotine, 1991). The control objective is to 

transform the vessel dynamics (3) into a linear system v = a , where a can be interpreted 
as a body-fixed commanded acceleration vector. Considering the nonlinear model of 
Equation (3), the nonlinearities of the controlled system can be cancelled out by simply 
selecting the control law as: 

T F eed.Lin. = M(fl)-a b +C(v).v + D(v).v + G(fi) 

Notice that the injection of this control expression in the nonlinear model of Equation (3) 

provides the desired closed loop dynamic v = a . The commanded acceleration vector a 
can be chosen by pole placement or linear quadratic optimal control theory, a described 
previously. The pole placement principle allows for selecting the system poles in order to 
specify the desired control bandwidth. Let A = diag{A lf ^ 2 '---'^} ^ e tne positive diagonal 
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matrices of the desired poles A\ . Let v^ denote the desired linear and angular velocity 
vector, and v = v-v c | the velocity tracking error. Then the commanded acceleration vector 
can be chosen as a Pl-contr oiler with acceleration feedforward: 

a b =v d -K p .v-K r |v(r).dr 

Choosing the gain matrices as K p =2A and K| =a 2 , as proposed in (Fossen, 2002), yields a 
second order error dynamics for which each degrees of freedom poles are in s = -4 ( i =l,...n ), 
thus guaranteeing the system stability. 

In (Silvestre et al., 2002), the authors propose an elegant method, called Gain-Scheduling, 
where a family of linear controllers are computed according to linearizing trajectories. 
This work is based on the fact that the linearization of the system dynamics about 
trimming- trajectory (helices parameterized by the vehicle's linear speed, yaw rate and 
side-sleeping angle) results in a linear time-invariant plant. Then, considering a global 
trajectory consisting of the piecewise union of trimming trajectories, the problem is solved 
by computing a family of linear controllers for the linearized plants at each operating 
point. Interpolating between these controllers guarantees adequate local performance for 
all the linearized plants. The controllers design can then be based on classic linear control 
theory. 

Nevertheless, these issues cannot address the problem of global stability and performances. 
Moreover, the reader has noticed that these methods imply that the model parameters are 
exactly known. In Feedback Linearization technique, a parameter misestimation will 
produce a bad cancellation of the model nonlinearities, and neglect a part of the system 
dynamics that is assumed to be poorly excited. This assumption induces conservative 
conditions on the domain of validity of the proposed solution, thus greatly reducing the 
expected performances, which in turn, cannot be globally guaranteed. 

The Sliding Mode Control methodology, originally introduced in 1960 by A. Filipov, and clearly 
stated in (Slotine, 1991), is a solution to deal with model uncertainty. Intuitively, it is based on the 
remark that it is much easier to control l st -order systems, being nonlinear or uncertain, than it is to 
control general n th -order systems. Accordingly, a notational simplification is introduced, which 
allows n^order problems to be replaced by equivalent l st -order problem. It is then easy to show 
that, for the transformed problems, 'perfect 7 performance can in principle be achieved in the 
presence of arbitrary parameters accuracy. Such performance, however, is obtained at the price of 
extremely high control activity. The basic principles are presented in the sequel. Consider the 
nonlinear dynamic model of Equation (3), rewritten as: 

ii P =F(f| Pl i|p)+H(i|p)-u 

where r| P is r\ expressed in Vessel Parallel Coordinate system {P}, as defined previously. F and H 
are straightforward-computable nonlinear matrices expressed from Equation (3), that are not 
exactly known, and F and H are their estimation, respectively. A necessary assumption is that 
the extent of the precision of F is upper-bounded by a known function F(f| P ,!|p), that is If - f| < F • 

Similarly, the input matrix H is not exactly known, but bounded and of known sign. The control 
objective is to get the state t] P to track a desired reference ijp (( j , in the presence of model 

imprecision on F and H. For simplification reasons, we consider in the following that the H matrix 
is perfectly known. For a detailed description of a complete study case, please refer to (Slotine, 
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1991). Let ij = r\ -n p d be the tracking error vector. Let s be a vector of a weighted sum of the 
position and the velocity error, defining the sliding surface s (t) . 

s=np +^i i{p 

where Xi is a diagonal matrix composed with strictly positive gains. With this framework, 
the problem of tracking r\ p =r\ p . d is equivalent of remaining on the surface S(t) , for all t ; 
indeed s(t) = represents a l st -order linear differential equation whose unique solution is 
ij =0/ given initial condition t| p (o) = t| p d (o). The problem of keeping the scalar components 
of s at zero can now be achieved by choosing the control law u such that, outside S(t): 

where ^ is a vector composed with strictly positive gains, s 2 is the vector composed with 

the squared components of s and Isl is the vector composed with the absolute values of 

the component of s. Essentially, the previous expression is called the sliding condition, and 
states that the square 'distance 7 to the surface, as measured by s 2 , decreases along all 
trajectories, thus making the surface S(t) an invariant set. The design of u is done in two 
steps. The first part consists in controlling the system dynamics onto the surface S(t), 
expressed as S = . Assuming that H is invertible, solving formally this previous equation 

for the control input, provides a first expression for u called the equivalent control, u eq , 
which can be interpreted as the continuous control law that would maintain S=0 if the 
dynamic were exactly known. 



Ueq =H M-F+rid-^i 1 -^] 



The second step tackles the problem of satisfying the switching condition, Equation (9), 
despite uncertainty on the dynamics F (for simplicity the input matrix H is assumed to be 
perfectly known), and consists in adding to U eq a term discontinuous across the surface 

S=0: 
u = u eq -H _1 -Vsign(s) 

where X3 is a matrix composed with strictly positive functions A3 j , and sign(s) denotes the 

vector where the i th element equals to +1 is S\ > , or -1 if S\ < . By choosing 1 3 s = A3 1 (t| P ,i|p ) 

to be Targe enough 7 , we can now guarantee that the sliding condition (9) is satisfied. Indeed, 
we obtain the expression: 

which is a negative definite vectorial expression if the functions A,3j(f|p ,i|p ) are chosen 
according to the choice of: 

^3 = F(ti P/ iip)+£(s) 
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where e(s) is a positive margin vector. This implies that the system will effectively converge 
towards the sliding surface, on which the trajectories are all converging to the origin. The 
consideration of a misestimation in the input matrix H, brings into the choice of the X 3 j the 

guaranteed upper and lower bounds H max and H min on the uncertainty of H, (Slotine, 1991). 
This control technique has been applied in various systems, and an application to the Taipan 
1 AUV is detailed in (Vaganay et al., 1998), where the authors uses the control robustness to 
compensate for the uncertainty of a linear equivalent controller u eq . The main drawback of 

this method is the extremely high control activity that generates the switching part of the 
control scheme, induced by excited unmodelled dynamics. Moreover, bounds on the 
components of s can be directly translated into bounds on the tracking error vector rj p , and 
therefore the components of s represent a true measure of tracking performance. 
Nonlinear control design, based on Lyapunov theory and Backstepping technique, allows for 
considering the full nonlinear model of the system and the model uncertainty in order to 
guarantee asymptotic performances of the controlled system. The problem and the model 
are decoupled according to the three planes of evolution, vertical, horizontal and transverse 
planes, where the control objectives are path-following, diving control and roll compensation. 
We illustrate in the following an application of this solution to the vehicle Taipan 2 2 ). The 
three simplified nonlinear models are written as: 

• Path-following in the Horizontal Plane 
The dynamic horizontal model is extracted from the model of Equation (3), as: 

F u =m u -ii + d u 

= m v -v + m ur -u-r + d v 0-0) 

r r =m r -r + m pq -p-q + d r 

where F u is the surge force induced by stern propellers and T r is the controlled torque induced by 
rudder or differential action of stern thrusters, m u =m-X L j is the system mass including inertial 

(m) and added ( X q ) masses along the surge direction, m v = m - Yy is the system mass in the sway 

direction and m r = I zz - N ? is the moment of inertia of the system around the z axis, including the 

intrinsic moment of inertia ( I zz ) and the moment of inertia induced by added mass ( N f ). 

m ur =m-Y ur and m pq =l yy are cross-coupling velocity terms and d u , d v and d r are 

hydrodynamic damping and restoring terms, whose expressions can be found in (Lapierre c et al., 
2006). X, Y, and N are the hydrodynamic derivatives composing the matrices of the model of 
Equation (3). From the control expression , the relevant parameters are m u , m r , m r , m ur , m pq , 

d u , d v and d r , and constitute the parameter vector p. 

The kinematic model is reduced to the rotation matrix R, expressing the following relations: 

x = u -cos(^)- v-sin(^) 

y = usin(^) + v cos(^) (11) 

ij,= r 

The chosen guidance system is based on the virtual target vehicle principles. As described before, 
this method requires parameterizing the desired path in function of a curvilinear abscissa s of a 



2 cf . Figure 10 in the paper Underwater Robots Part I : current systems and problem pose. 
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point P(s), moving along the path, and defining an approach angle 8 as described in Equation (6). 
The coordinates of the point P are expressed as [s x y x f in the body-frame {B}. Equipped with this 

formalism, the control objective can be expressed as follows: 

Consider the AUV model with kinematic and dynamic equations given by (3) and (11), respectively. 

Given a path to be followed, a desired u^ > u m j n > for the surge speed u, and a set of reasonable 

estimation of the parameter set P , derive feedback control laws for the force F u , torque T r , and rate 

of evolution S of the curvilinear abscissa s of the "virtual target" point P along the path so that y l7 

Si , y/ , and u - Ud tend to zero asymptotically. 

This problem decomposed in three sub-objectives. 

i Global Uniformly Asymptotic Convergence (GUAC) of the kinematic level. 

Expressing the kinematic model in the body frame, and considering the total 
velocity of the system v t = (u 2 + v 2 y and the side-slip angle J3 = arctan(v / u ) , yields: 



(12) 



In the kinematic case, the control input is reduced to r, the yaw-rate, and u is 
assumed to be equal to Ud. Recall that the virtual target principle requires to 
define the approach angle 8(\/i), given in Equation (6), and that the guidance 

objective is to make the angle 6 tracking S(yi). This sub-problem is solved in 
considering an appropriate Lyapunov candidate function V x =^-(<9-^(y 1 )) , and 
extract control r satisfying the condition that M 1 -\/\ < if * 8 . Then, verifying that 
Vi is bounded complete the requirements of the application of the Barbalat's 
lemma (Khalil, 2002) which allows proving that the sub-set defined as the system 
trajectories where the approach error is null, Q x :=[ L 9-^(y 1 ) = 0] is invariant. Then, 

the analysis of a second Lyapunov candidate V 2 = /^(si 2 + Yi 2 )/ restricted onto £l\ , 

and the use of the LaSalle's theorem (Khalil, 2002) show the implicit consecutive 
convergence of the system toward the path, if the system is autonomous, that is if 
the desired forward velocity Ud is time invariant. This is stated in the following 
proposition: 

Consider the robot model (10) and (11) and let a desired approach angle be defined by Equation 
(6). Further assume that measurements of [u v] T are available from robot sensors and that a 
parameterization of the path is available such that: given s , the curvilinear abscissa of a point 
on the path, the variables , yi , S\ and c c (s) are well-defined and computable, where 

c c (s) = denotes the path curvature at s. Then the kinematic control law: 



1 '^iL.^L-k^-^+c^.S 



1-IDuL. .COS 2 /? I V, 2 m 

m r 
s = v t •cos^ + k 2 -Sj 



(13) 
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applied to a stern-dominant vehicle, drives 6 , yj and Sj asymptotically to zero, with kj and 
k 2 2 arbitrary positive gains, and given the initial relative position \o, Si , yi ] t=0 • That is, the 

kinematic model of the AMY is asymptotically and uniformly converging to the desired path 
Notice the 'stern dominant 7 condition is required in order to insure that: 

m ML= m-Y ML<1 
m v m-Yv 
It is interesting to notice that this condition has been stated in (Lewis*, 1988), 
according to different consideration, and is related to the open-loop stability of the 
system, restricted to the horizontal plane, 
ii. Global Uniformly Asymptotic Convergence (GUAC) of the dynamic level. 

The above feedback control law applies to the kinematic model of the AUV only. 
However, using Backstepping techniques (Krstic, 1995), this control law can be extended 
to deal with vehicle dynamics. In the kinematic design the total velocity v t (t) of the 

vehicle was left free, but implicitly dependent on a desired time-invariant profile u <j for 
surge speed u(t). In the dynamic design the variable u will be brought explicitly into 
the picture and a control law will be derived so that u d -u(t) tends to zero. Notice also 

that the robot's angular speed r was assumed to be a control input. This assumption is 
lifted by taking into account the vehicle dynamics. The following result holds. 
Consider the robot model (10) and (11), and the corresponding path following error model in (12). 
Let a desired approach angle be defined by Equation (6) and let the desired speed profile 

u d >u min >0 • Further assume that measurements of [u v rj are available from robot 
sensors and that a parameterization of the path is available such that: given s, the curvilinear 
abscissa of a point on the path, the variables 6 , y^ , Sj and C c (s) are well-defined and 
computable. Then the dynamic control law: 

r r = m r -a r - d r 
<F U = m u -(u d -k 4 (u-u d )-d u ) (14) 

s = v t -cos^ + k 2 -Si 



where 



f« -k 3 -(r-r d )-k 5 .(0-£) 



■^•cos 2 /? 



r -6 i (a A ■■ . ii'-v 2-v t -/? u m ur . d v 

V t 2 v t V t 2 ^ m v m v 

r d =S-j3-k r {0-S)+z c {s).s 
and k. , for i=l,...5, are arbitrary positive gains, and given the initial relative position 
F' s l»yi]t=0' drives the system dynamics in order for 6 , y^ and Sj to asymptotically and 
uniformly converge to zero, assuming a perfect knowledge of p. 
This solution is derived according to the consideration of the Lyapunov candidate 

function V 3 = ^2- (r -r^ ) +%•[[} -U^) , capturing the convergence properties of the 
system yaw rate to the kinematic reference r^ , which is a rewriting of the kinematic 
control solution previously exposed. Using same type of argument than used for the 
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kinematic case, it can be shown that this convergence requirement induces the 
dynamic model of the system to asymptotically and uniformly converge to the path. 
For a complete proof, please refer to (Lapierre a & Soetanto*, 2006) and (Lapierre b * et al., 
2003). 
iii. Robust Global Uniformly Asymptotic Convergence (GUAC) of the dynamic level. 

This section addresses the problem of robustness to parameters uncertainty. The previous 
control is modified to relax the constraint of having a precise estimation of the dynamic 
parameter vector P, by resorting to backstepping and Lyapunov-based techniques. Recall that 
the kinematic reference expression involves the estimation of the horizontal model dynamic 

parameters. Let r op (P) be the kinematic control law computed with the exact dynamic 
parameters, as expressed in Equation (12), and let r d (p) be the evaluation of this control 
expression, considering the approximated value of the parameters. It is straightforward to 
show that the neglected dynamics P , induces a non negative derivative of the V i Lyapunov 
candidate, y\ = -k : • (<9 - sf + (6 - s) ■ Ar , where Ar = r opt - r A d . 

The design of the dynamic control is done as previously, considering the Lyapunov 
candidate V 3 = y • (r - f d ) 2 + y 2 • (u - u d ) 2 . The resulting control is expanded in order to 
make explicitly appear the parameters, and results in the following affine expression: 



i=l 
11 



(15) 



fr +X qi " 9i ~ Q "4~ 



j=l 



l-q 3 -g 3 

where pj (P) and C|j(P) (i =1,...,11 and j =1,2,3) express groups of the system dynamics 
parameters, and fj(n,v), g j (t| 7 v) and f r (t| f v) are functions dependant on the system states. 

Let Apj = Pj (P)- Pj (Pj and Aq j = q j (p)- q A j (p) be the estimation error in the evaluation of the 
parameters Pj , involved in the control previous control expression. The misestimation Apj 
induces V 3 to be non negative-definite, as: 

V3 = -k 3 -(r-r d ) 2 -k 4 -(u-u d ) 2 + AV3, 
where 



AV '3-Z( r " rAd )" APi " fi+ Z( u_Ud ^ APi " fi ■ 

1=1 1=8 

11 2 

Then the consideration of the Lyapunov candidate v = y ■ + y .V* P' leads to pose: 



i-i 



362 Mobile Robots, Towards New Applications 

Pi =-V(r-r A d )- f,,fori=l 7 (i6) 

Pi =-A i -(u -u d )- f j , for i =8,. ..,11 

in order to obtain V 4 < . This is done implicitly using the fact that Apj = Pj , since pj (p) is 

assumed to be static. This parameter adaptation is possible since the parameters Pj appear 

in an affine form in the control expression. Recall that the control (15) has been designed to 
drive the system toward the reference r d , whose expression involves an estimation of the 

some of the dynamical parameters. As a consequence, the control (15) with the adaptation 
scheme (16) drive asymptotically the system toward the non-exact kinematic reference r d . 
In order to recover the GUAC requirement, the parameters (\\ need to be adapted. The 

adaptation design previously-used cannot be applied since the nonlinear form in which the 
parameter q3 appears. Then, the robust scheme is developed relying on switching system 

theory, applied to control design, as stated in (Hespanha et al., 1999). Concerning marine 
systems and our system of Equation (15), the robust scheme relies on the choice two 
different values q| P and q J ™" , guaranteed to overestimate and underestimate the real 

value q i ( Aq ■ p > and Aq ■ iown < ), and use them according to: 



q i = 



qs 



[qy p if(0-<?).g, <0 
q? own elsewhere 

r ( 17 ) 

|q3 up if(£-4g 3 .r<0 

I q 3 down elsewhere 



Using the fact that g 3 > and 1— P3 -93 >0, it can be shown (Lapierre b & Soetanto, 2006), 

that the Lyapunov function V^ = [6 - S) , restricted to the invariant set Q r := [r - r d = 0] has 
a following negative expression: 

' 2 

^Aq| -gj + Aq 3 -g 3 -r 



Vi =■ 



I-Qb-Qb 



j=i 



<0 



J 



At last, we can state the final proposition: 

Consider the robot model (10) and (11), and the corresponding path following error model in (12). Let 
a desired approach angle be defined by Equation (delta) and let the desired speed profile 

u d >u min > • further assume that measurements of [u v r] are available from robot sensors 
and that a parameterization of the path is available such that: given s, the curvilinear abscissa of a 
point on the path, the variables 6 , y\, Si and C c (s) are well-defined and computable. 
Consider that a reasonable estimation of the model parameters P, is used to compute the 11 initial 
values P| and the 3 pairs q^ and q j 0wn such that q' 0wnn < qj < qj P . Then the control law (15), 

with the adaptation scheme (16) and the switching scheme (17), and given the initial relative position 
\6, S]_, Yi ] t= o / solves the robust path-following problem. 
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• Vertical plane 
The dynamic vertical model is extracted from the model of Equation (3), as: 

F u =m u -u + d u 

F w =m w -w + m uq -u -q + d w (18) 

r q =m q -q + m pr • p-r+d q 

where F u is the vertical force induced by the common action of the stern and bow control 
surfaces and Y q is the torque applied around the sway direction, induced by the differential 
action of the control surfaces. m w = m-Z w is the system mass including inertial (m) and 
added mass (Z^ ) along the heave direction and m q = I yy - M ^ is the moment of inertia of 
the system around the y axis, including the intrinsic moment of inertia ( I yy ) and the 
moment of inertia induced by added mass ( M q ). m uq =-m and m pr =-l zz are cross- 
velocities coupling terms and d w and d q are hydrodynamic damping and restoring terms, 

whose expressions can be found in (Lapierre c et al., 2006). X, Z, and M are the hydrodynamic 
derivatives composing the matrices of the model of Equation (3). 

A similar strategy than exposed previously for the horizontal plane is used detailed in (Lapierrec et 
al., 2006). Nevertheless, considering an AUV with bow and stern control surfaces (case of the 
vehicles Tarpon 2 3 and Infante, ISR, Lisbon Portugal), the system model is fully actuated, assuming 
a strictly positive forward velocity. Indeed, an AUV only carrying stern control surfaces (case of 
the vehicles Remus - Hydroid Inc., Cape Cod, MA, USA - and Gavia - Hafmynd, Reykjavic, Island) 
cannot be actuated along the z axis. Moreover, the UUV systems are generally positively buoyant, 
for safe recovery reason. This induces on the system a constant upward force, which requires to be 
compensated in pitching negatively. Then, such a system cannot insure a precise depth control 
with a null pitch angle. In opposition, a fully-actuated AUV in the diving-plane, is capable of 
changing depth with a constant (even null) pitch angle, and compensate the positive buoyancy 
during constant pitch and depth control. This makes these systems particularly well-suited for 
shallow-water or near-bottom applications, where a precise depth and pitch control is required. 
From the control point of view, the designer is facing an over-constrained problem, where two 
strategies exist for changing depth. This trade-off is solved at the guidance level in combining both 
strategies according to the system situation. An obstacle avoidance manoeuvre, or a change in the 
bottom profile, may induce a rapid reaction, and the pitch strategy will be preferred. On the other 
hand, a precise survey is requiring a null pitch angle, and a smooth evolution of the distance to the 
bottom can be compensated without modifying the system attitude. A solution to this problem is 
detailed in (Lapierre c et al., 2006). The adaptation design benefits from the fully-actuated nature of 
the system, and classic continuous method (in opposition to switching) works. Nevertheless, the 
work in (Lapierre & Jouvencel*, 2006) underlines a major drawback of the method. That is the 
evolution of the parameters estimate ( pj in the horizontal case), is not guaranteed to be bounded 
in presence of sensor noise. Then, the authors propose to design a robust switching scheme (as 
done for the parameters q ■ in the horizontal case) for all the system parameters. This leads to a 

discontinuous evolution of the control inputs, and the effect on the actuator activity has to be 



3 cf . Figure 10 in the paper Underwater Robots Part I : current systems and problem pose. 
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explicitly studied. Notice the analogy of this solution with the sliding-mode control design as 
exposed previously. This warrants further research on this topic. 
• Transverse plane 

= rriv -v+ m ur -u -r + d v 

F w =m w -w + m uq -u-q + d w 

r p = m p ' P + d p 

where T p is the torque applied around the surge direction, induced by the propeller rotation. 
m p = l xx -K p is the moment of inertia of the system around the x axis, including the intrinsic 

moment of inertia ( | xx ) and the moment of inertia induced by added mass (L). d is the 

hydrodynamic damping and restoring term, whose expressions can be found in (Lapierre & 
Jouvencel*, 2006). 

The control objective in the transverse plane is to keep the rolling angle $ as small as 
possible. Indeed, the decoupling condition is that <p « . Considering the symmetrical shape 
of the system, the principal excitation of the rolling dynamics is due to the stern thrusters' 
activity, which for a given motor regime induces a rolling torque. This compensation is 
generally statically performed with static fins placed in the propeller flow. 

5.4 Biological Inspired systems. 




Fig. 9. Swimming movement from (a) anguiliform, through (b) sub-caranguiform and (c) 
caranguiform to (d) thunniform mode, from (Lindsey, 1978). 

All type of vertebrate locomotion relies on some kind of rhythmic activity to move forward: 
undulations or peristaltic contractions of the body, and/ or oscillations of fins, leg or wings. 
By rhythmically applying forces to the environment (ground, water or air), reaction forces are 
generated which move the body forward. This type of locomotion is in contrast to the most 
man-made machines which usually rely on few degrees of freedom (e.g. a limited number of 
powered wheels, propellers, or jet engines), and continuous, rather than rhythmic, actuation. 
From a technological point of view, animal locomotion is significantly more difficult to 
control than most wheeled or propelled machines. The oscillations of the multiple degrees of 
freedom need indeed to be well coordinated to generate efficient locomotion, (Ijspeert, 1995). 
These oscillations are produced by the system actuation, controlled to follow a desired body- 
shape deformation profile, called locomotive gait. The choice of the locomotive gait is a crucial 
question. In the case of an eel-like robot, this body-shape evolution is a propagating wave, 
adjusting the signal phase according to the joint situation located on each vertebra. Indeed, a 
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badly chosen gait will agitate the robot without guaranteeing that the maximum number of 
system elements (vertebrae) is involved in the thrust generation, loosing by the way the 
desired efficiency in terms of propulsion and manoeuvrability. Biologists have identified 
different types of gait in function of the animal, from undulatory to oscillatory movement 
(this study does not consider locomotion induced by fins movement), cf . Fig. 9. 
Using in vivo observations, E. Tytell, in (Tytell, 2003) proposes a generic mathematical 
expression of the gait profile, expressing the lateral (y) position of the animal midline. 

y(s e )=A.e^ /L - 1 ).slnf^.(s e -V.t)l (19) 

where S e is the contour length along the midline starting at the head, A is the tail beat amplitude, 
a is the amplitude growth rate, L is the body length, X is the body wave length, t is time and V 
the body wave speed. By this definition, a large CC implies that amplitude is low near the head 
and increases rapidly near the tail. A smaller a implies more undulation anteriorly. This author 
proposes also an adaptation profile of the parameter a at a given speed. 
From a robotics aspect, the motion control problem can be divided in three questions. 

i . The system ability to produce a thrust according to a desired forward velocity. This 
question underlies three points. 

- The gait choice. 

- The control of the joint actuation in order to follow the previous reference. 

- The adaptation of the gait parameters according to the system situation. 
i i . The control of the system heading according to a desired reference. 

i i i . The combination of both the previous solutions according to path following requirement. 
The literature proposes some actuation gaits, directly controlling the inter-vertebrae 
actuation as a trajectory tracker. In (Mclsaac & Ostrowski, 2002), the following actuation 
sinusoidal gait has been chosen. 



<P\ 



(t)=A.sin(ffl.t + (i-2).p s )+p off (20) 



where <p\ is the joint angle of the articulation i, t is the time, A maximum amplitude of the 

oscillation, co is the wave frequency, cp s is a constant phase difference between links, and cp ff is 

the parameters that acts on the system angular velocity. Clearly, this choice is a sub-expression of 
(19), with the difference that (20) expresses a temporal joints evolution, when (19) describes the 
body shape evolution. Indeed, despite the mathematical expression similarity, nothing guarantees 
that a perfect tracking of the joint reference (20) induces for the system to geometrically converge to 
the shape described in (19). Saintval, in (Saintval, 2002), proposes an optimal approach to find the 
best parameter's gait for a three-links robotic carangiform prototype. As underlined by G. Gillis 
(Gillis, 1998), natural eels are adapting the gait parameters in function of the forward speed. For 
examples, the analysis of the observations done by G. Gillis shows clearly that tailbeat frequency 
increases significantly with swimming speed. Moreover this relation tailbeat frequency / forward 
speed appears to be linear, this results is also observed by D. Webber et al, in (Webber et al, 2001). 
The work reported in (Lapierre & Jouvencel, 2005) and (Parodi et al, 2006) started a study in order 
to design a closed loop control of the serial chained system (representing the eel) that adapts the 
gait profile in function of the system situation in order to increase performance. The starting point 
comes again from a biological inspiration, stating that fishes are using lateral line sensors to 
monitor the surrounding flow field for manoeuvring underwater, (Fan et al, 2002). This pressure 
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sensors line allows fishes to locally control the fluid flow quality along their body. This 
observation gives a very interesting intuition on how closing the control loop in order to improve 
the system performance. The adopted methodology is following the 6 following points. 

i. Designing an autonomous gait: literature proposes actuation gaits which are 
explicitly time-dependent. Moreover, the shape deformation is induced directly by a 
joint trajectory tracker. This is the classic problem of path following versus trajectory 
tracking. As shown for the case of nonholonomic wheeled robot [25] or 
underactuated marine craft, a path following controller achieve smoother 
convergence to a path, when compared to trajectory tracking control law, and the 
control signals are less pushed to saturation. Moreover, from a theoretical aspect, 
powerful mathematical tools are available for autonomous system, which are not 
usable in the case of non autonomous systems, (Lapierre & Jouvencel, 2005). 

ii. Controlling the curvature profile: this allows for controlling effectively the body 
shape, instead of applying directly the actuation gait. The analogy with the gaits 
coming from biological observation is direct. Moreover, this method allow for closing 
the control loop on the system states, resulting in an efficient system regulation, 
(Lapierre & Jouvencel, 2005). 

iii. Adapting the gait parameters in function of the system situation: the previous 
theoretical framework permits the design of an adaptive scheme, improving 
performance, (Parodi et at., 2006). 

iv. Coupling with path following requirement: the gait parameters are modified in order to 
guarantee the system to reach and follow a desired path, (Lapierre & Jouvencel, 2005). 

v. Coupling with a local force control: this item concerns an active open issue. Notice that the 
previous solutions (autonomous or not) are closing the control loop on an arbitrary 
reference, resulting in a shape control without explicit connection with the fluid. The 
solution consists in combining the previous solution with a local force control requirement 
in order to adjust the previous gait and improve system performances. 

vi. Designing a pure force control: the final objective is to design a pure force control of 
the system, in order to guarantee that each system element takes part in the thrust 
generation. This has to be done in embedding the previous curvature-based gait 
control as a model-based guidance function in order to drive the force control around 
the desired gait. Another advantage of this hypothetical solution is that the necessary 
measurement inputs will be restricted to current driving the DC motors of the system. 
This warrants further research on this subject. 

6. Mission Control 

Among the challenges that face the designers of underwater vehicle systems, the following is of 
major importance: design a computer-based mission control system that will, (Oliveira, 1998): 
i. enable the operator to define a vehicle mission in a high level language, and translate 

it into a mission plan, 
ii. provide adequate tools to convert a mission plan into a mission program that can be 

formally verified and executed in real-time, 
iii. endow an operator with the capacity to follow the state of progress of the mission 
program as it is executed, and modify if required. 
The first point concerns the aggregation of the different elementary actions the robot is 
able to perform, in higher level primitives that the mission designer is organizing in 
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function of the mission requirements. Moreover, vehicles have to be programmed and 
operated by end-users that are not necessarily familiarized with the most intricate 
details of underwater system technology. Generally, the framework adopted builds on 
the key concept of Vehicle Primitive, which is a parameterized specification of an 
elementary operation performed by the vehicle. Vehicle Primitives are obtained by 
coordinating the execution of a number of concurrent System Tasks, which are 
parameterized specifications of classes of algorithms or procedures that implement 
basic functionalities in an underwater robotic system. Vehicle Primitives are in turn 
logically and temporally chained for more abstract Mission Procedures, which are 
executed as determined by Mission Programs, in reaction to external events. 
At a second level, Vehicle Primitive can be embodied in a Petri net structure, analysed in 
order to extract the system properties. The determinism of the Petri net structure can then be 
guaranteed, in order to meet the desired performance requirements. Another necessary 
stage is the development of special software environments that provide the necessary tools 
to go through all the stages of mission programming, followed by automatic generation of 
target code that runs on the vehicle computer network. 

The third point is related to the on-line Man-Machine Interface (MMI) that manages the 
communication between the machine and the operator, and requires covering different 
aspect, according to the user objectives. The system designer has to access to low level 
information in order to test and validate the material and software architectures. The control 
developer has to integrate and tune its solutions, in terms of navigation, guidance, control, 
path planning... The low level mission controller designer is proposing to combine different 
system tasks in order to implement new vehicle primitives. The end-user has to program the 
mission in understandable terms according to its scientific speciality. And at last, the MMI 
has to include the on line system supervision module according to the permanent 
communication capabilities, that should allow for in-line mission replanning. 

7. Software and Hardware Architectures 

The previous solutions are reliable if the system is equipped with a software architecture 
that allows for deterministically running the previous algorithms onboard the vehicle. 
Moreover, notice that the acoustic devices the vehicle is carrying are potentially interfering. 
The management of the sensors' recruitment is a low level problem that has to be explicitly 
considered, in order to guarantee the reliability of the measurements. This problem belongs 
of the software engineering field, and follows three main approaches. Deliberative or supervised 
architectures are based on planning and allows for reasoning and making prediction. 
Behavioural or reactive architectures allows for the system to continuously reacting to the 
situation sensed by the perception system. Finally, hybrid architectures are complementing 
the advantages of the two previous ones, limiting their drawback. Comparative studies have 
been reported in the literature with (Ridao et al., 1999), (Valanavis et al., 1997), (Coste- 
maniere et al., 1995) and (Arkin, 1998). Moreover, the determinism requirement imposes to 
deploy the previous algorithms on a real-time operating system (Linux-RTAI, QNX, 
Vx Works, Windows-RTX, MicroC/OS-II...) guaranteeing the execution time of the 
processes. The latency of computing, or induced by the Local Area Network (LAN) 
communications, should be explicitly considered from the control aspect. Nevertheless, this 
highly complex problem is generally neglected in over-dimensioning the electronics and 
medium capacity. 
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From the hardware aspect, the objective is to design a reliable technological target onto 
which all the previous solutions are uploaded. The increase in hardware computing 
capabilities and the decrease in power consumption makes possible to use distributed 
computing architecture onboard underwater systems, where consumption, reliability 
and robustness are key-problems. A distributed system can be composed by many 
processes running on different computers, working together to meet the desired 
functional requirements of the mission control system. The distributed processes are 
coordinated using Inter-Processes Communication (IPC) and synchronization 
mechanism that include semaphores, Fifos, mailboxes, etc. The benefits of distributed 
processing are (Alves et ah, 1999): 

i. increased performance by executing processes in parallel, 

ii. increased availability, because a process is more likely to have a resource available if 
multiple copies are available, 

iii. increased reliability since the system can be designed to be faul-tolerant and then 
automatically recover from failures, 

iv. increased adaptability and modularity, since parts can be added or removed, 

v. expensive resources can be share. 
Asynchronous processes communicate with each other by exchanging messages. A common 
approach to distributed processing uses Local Area Network (LAN) to connect group of nodes 
together. Every node in the network can work alone as well as communicate with other nodes 
over the network to transfert data and/ or synchronisation messages. Then, an application running 
on a local node can use resources of a remote node. Ideally, in distributed processing 
environments, the access to remote resources should be transparent. Different network topologies 
and communication strategies can be used to connect the different nodes of the architecture. For 
more information on this subject, please refer to (Alves et ah, 1999), (Halsall, 1996) and (Bennet, 
1994), and for specific application on underwater systems, (Caccia et ah, 1995). 

8. Conclusion 

This paper surveyed the current state-of-the-art in the area of the underwater robotics, 
focusing on the Modelling, Navigation, Guidance and Control aspects. The problems of 
Mission Control and Software Architecture are key questions belonging to the Software 
Engineering domain and were briefly mentioned. Adding the Hardware Architecture 
question, these 7 points constitute the set of sub-problems, to be solved according to specific 
criteria and design methods, in order to realize a reliable system with guaranteed 
performances. The diversity of missions in which UUVs can be used, necessitates 
developing modular and reconfigurable vehicles, equipped on demand, and the capability 
to take place in a flotilla where the collaborative work brings new constraints. 
Underwater robotics involves a complete set of theoretical problems, warranting long hours 
of exciting investigations. Moreover, the recent needs have pushed the development of new 
systems affording users advanced tools for ocean exploration and exploitation. The domain 
of underwater robotics is intended to grow rapidly, and the time where autonomous marine 
vehicles will freely roam the ocean is imminent. 

As for any robotics application, one of the most important questions concerns the system 
reliability, in order to provide the necessary confidence before letting these vehicles go on 
populated routes. This problem impacts on the entire system, and we presented some 
solutions in order to guarantee the system performances. 
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Nevertheless, besides the robotics aspect, various problems have to be simultaneously 
considered, from network communication capabilities (between vehicles or infrastructure), 
via the energetic and sensing aspects, to the adaptation of the maritime legislation and 
public acceptation. These last two points may be the most difficult ones, but guess that the 
importance of the related issues will motivate the decision makers to provide rapid answers, 
and support emergent solutions. 
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1. Introduction 

The underwater survey and inspection are mandatory step for offshore industry and for 
mining organization from onshore-offshore structures installations to operations (Whitcomb, 
2000). There are two main areas where underwater target tracking are presently employed for 
offshore and mining industry. First, sea floor survey and inspection and second is subsea 
installation, inspection and maintenance. This paper takes second area into account and AUV 
vision system is developed that can detect and track underwater installation such as oil or gas 
pipeline, and power or telecommunication cables for inspection and maintenance application. 
The usage of underwater installations has increased many folds. It is desirable to do routine 
inspection and maintenance to protect them from marine traffic, such as fishery and anchoring 
(Asakawa, et al., 2000). Detection and tracking of underwater pipeline in complex marine 
environment is fairly difficult task to achieve, due to the frequent presence of noise in a subsea 
surface. Noise is commonly introduced in underwater images by sporadic marine growth and 
dynamic lighting condition. 

Traditionally, vigilances, inspections and maintenances of underwater man made structures 
are carried out by using the remotely operated vehicle (ROV) controlled from the mother 
ship by a trained operator (Whitcomb, 2000). The use of ROV's for underwater inspections 
are expensive and time consuming job. Furthermore controlling the ROV's from the surface 
by trained operators required continuous attention and concentration to keep the vehicle in 
the desired position and orientation. During long mission, this become a tedious task and 
highly prone to errors due to lack of attention and weariness. Moreover, tethering the 
vehicle limits both the operation range and the vehicle movements (Ortiz, 2002). The 
autonomous underwater vehicle's do not have such limitation and essentially present better 
capabilities to those of ROV's. AUV's have a wider range of operations as there is no 
physical link between the control station on the surface and the vehicle, as they carry their 
power supply onboard. The usage of AUV for underwater pipeline or cable inspection and 
maintenance become very popular area of research for mining and offshore industries 
(Griffiths & Birch 2000). During the last decade lots of efforts have been done for design and 
development of different AUV tracking system, to do routine inspection and maintenance 
for underwater installation (Asif and Arshad 2006). Conventionally, the literatures on 
underwater pipeline or cable tracking can be categorized according to the sensors used for 
detection and tracking. There are mainly three types of sensors which used for that purpose. 
The first two types of sensors are the sonar and the pair of magnetometers (Petillot, et al., 
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2002; Evans, et aL, 2003; Balasuriya & Ura 1999). These sensors provide effective tracking 
and successfully used in various tracking system. However, the problems with sensing 
devices are the power consumptions and the size of the devices itself. Furthermore, the 
measurements obtained using these sensors are extremely sensitive to noise. The third and 
most commonly used sensor for detection and inspection is the optical sensor or vision 
systems. The video camera mounted on a vehicle provides lots of information that can be 
examined by on board vision processing unit for effective path planning and navigations. 
Video camera is a high resolution sensor and is invaluable in situation required accurate 
measurements at short ranges. Generally, detection and tracking of an object in the natural 
marine environments using vision system presents several challenges (Ortiz, 2002). Due to 
the properties of ocean medium, optical waves are rapidly attenuated. Back scattering 
caused by marine snow, which are the presence of floating organic or inorganic particles in 
water which reflect light and degrades the visibility conditions. 

Recently, several approaches to underwater pipeline tracking have purposed utilizing 
different characteristics such as 3D or 2D underwater pipeline or cable models (Balasuriya & 
Ura 1999; Foresti, 2001) and computational methods like template matching, Hough 
transform, neural network, standard or extended Kalman filter (Foresti, 2001). 
Conservatively, underwater tracking systems either based on feature based technique or 
they used underwater pipeline or cable model for detection and tracking in an image 
sequences. In feature based technique, tracking is performed by using the low level features 
such as pipeline boundary edges (Matsumoto & Ito 2002; Balasuriya, et aL, 1997; Zanoli & 
Zingretti 1998). However, this technique may fail in case of occlusion due to growth of 
underwater plants or due to mud or sand on underwater pipeline or cable. On the other 
hand, model based approach based on prior knowledge or model of underwater pipeline 
such as straight line or structure of the underwater pipeline or cable (Balasuriya & Ura 1999; 
Foresti, 2001). The features extracted using the various image processing technique are 
matched with prior model. This prior model can be seen as a regularization term in 
measurement process. In this way model based method is robust against noise and missing 
data due to occlusion. 

This paper purpose a model based approach to detect and track underwater pipeline in 
complex marine environments. The object of this research paper is to design and implement 
a vision guidance system for autonomous underwater pipeline tracking and navigation. The 
purposed vision system use unconventional gray scale conversion technique to enhance the 
image and then Perona Malik filter is used to reduce the noise effect and enhance the 
features of underwater pipeline. To detect the pipeline boundary in an image, Hough 
transform is used. After detecting the pipeline in an image, parameterized curve is used to 
represent the underwater pipeline and for feature extraction. Based on extracted feature, 
curve fitting is used to measure the current pose and orientation of underwater pipeline. In 
order to track the underwater pipeline over time in an image sequence, the pipeline tracking 
problem is formulated in terms of shape space model. Shape space model is a mathematical 
relation used for describing the state of underwater pipeline or cable. To estimate the state 
of underwater pipeline over time Kalman filtering is used. 

The rest of paper is organized as follows: section 2 will presents the various image 
processing techniques that used for object detection in marine environments. Section 3 will 
discuss the methods for underwater pipeline modeling and shape space transformation. 
Section 4 will explain the feature extraction method and curve fitting technique. Section 5 
will discuss the pose and orientation measurements for autonomous navigation. Section 6 
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will presents the dynamic modeling technique and Kalman filtering method for underwater 
pipeline tracking. Section 7 will discuss the results obtained by testing the purposed system 
on real underwater images and finally section 8 will end the paper with conclusion and 
future works. 

2. Image processing 

After the exploration of potential of vision sensor most autonomous vehicles now used 
onboard vision sensor for control and navigation. It can provide measurement relative to 
local objects. However vision sensor required special image processing techniques to detect 
object and surrounding environments in image or in image sequences. The image processing 
techniques implemented in this project are outlined in Fig. 1. The brief discussions on these 
techniques are followed. 
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Fig. 1. Flow diagram for image processing. 

2.1 Color to gray scale conversion 

Conventionally the AUV is equipped with the color CCD camera to perform variety of task. The 
images acquired from the camera are in 24 bit RGB (Red, Green, and Blue) format. Colors are very 
important feature for any visual tracking system. However as the underwater installation gets 
older, corrosion, marine flora, and mud on underwater cables or pipeline modify this feature. So, 
as a first step of image analysis the images acquired by the onboard video camera are broken 
down to 8 bit values. There are several methods used to convert an RGB images into the grayscale, 
such as mean grayscale conversion, NTSC television standard and Intel image processing library 
formula. These types of conversion are suitable for several other applications such as television 
broadcasts and image editing and not very important for autonomous application. With this in 
mind different RGB channels are analyzed separately to enhance the images and extract boundary 
information of underwater pipeline effectively. Fig. 2 shows the individual analysis for red, green 
and blue channel. After doing series of experiments on real underwater images it found that the 
result of the red channel is well suited compare to green and blue channels. On the basis of these 
analyses only red channel is use for further processing. 
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Fig. 2. Result of converting color image into gray image by extracting only the (a) Red, 
(b) Green and (c) Blue channel. 
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2.2 Image filtering 

After converting image into the grayscale the next step is the image filtering. Due to 
the dynamic nature of lighting in the marine environment, images are often corrupted 
by noise. The existence of noise in an image affects the feature extraction step. As noise 
results in false edges being detected that may not exist and should not exist in the 
representation of the feature in the image. To overcome this problem, noise across the 
image is minimized by implementing smoothing or filtering operation Conventional 
smoothing techniques, like Gaussian, etc. are isotropic in nature and smooth the whole 
image in a similar fashion in all direction. Therefore while achieving the desired 
minimization of noise and variation across the image, the edges of the object in an 
image can loose contrast with the background. Further they also lead to the loss of the 
location of the actual boundaries. To cope up with this problem and to improve the 
image quality, the Perona-Malik (PM) filter (Perona & Malik 1990) has been selected 
for image filtering. The PM filter is anisotropic filter that retains the much needed 
edges information which is essential in detecting the pipeline boundary edges. It also 
improves the quality of the edge map obtains after edge detection, as edges are no 
longer produced around the noisy pixel regions. In Perona Malik filter, initial image I 
is modified using the anisotropic diffusion equation shown in (1). 

dI t =div(g(\Vl\)VI) (1) 

where / is the original image, dl t is the partial derivative of / with respect to diffusion 
time t, 'div' denotes the divergence operator, | VI | is the gradient magnitude of /and 
g | VI | is the diffusion coefficient diffusivity. The g(x) is a nonnegative monotonically 
decreasing function with g(0)=l and tend to zero at infinity so that the diffusion process 
will take place only in the interior. It will not affect the edges where the magnitude of 
the gradient is sufficiently large. The diffusivity function purposed by (Weickert, 2000) 
is used in this project as in (2) 

f 1 if l v/ G | = ° (2) 

, -3.15 ^ .„,„_, „ 
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where K is the threshold level for removing noise. The value of K plays very important role for 
smoothing the image without affecting the object edges and must be evaluated using the 
experimentation. This diffusivity function performs better in preserving/ enhancing edges. To 
implement (1) a finite based approach is used, because it is comparatively straightforward to 
implement for digital images. Equation (1) then can be discretized using the four nearest 
neighbors (north, south, east and west) and the Laplacian operator are given by: 

7*; 1 =I? J +A[C N V N I + C s V s I + C w V w I + C E V E I]l J (3) 

where 

c N = g(\v N i u \ 

and similar for south, east, and west. Finally Fig. 3 shows the result of PM filter on real 
underwater images. 
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Fig. 3. Result of Perona-Malik filter with 10 and 20 iterations. 

2.3 Pipeline detection 

The next phase of image processing is the detection of pipeline boundary. Before detection 
of object boundary, edge detection is performed to convert gray scale image into the binary 
image. Since the image quality is already improved using the grayscale conversion and PM 
filtering, Sobel edge detection is used to avoid the computational burden. 
Once image is converted into the binary, parameterized Hough transform is used to detect 
pipeline contour. The parametric equation of Hough transform is given below: 



p = xqos9 + ys\n9 



(4) 



At first all edge points are transformed into the Hough space using (4). In order to avoid the 
computational burden and excessive memory usage of Hough transform, 1000 edge pixels 
are processed at a time. After transforming all the pixels in Hough space, peak detection is 
performed and the locations that contain the peaks are recorded. To avoid the quantization 
problem in Hough transform all the immediate neighborhood of the maximum found 
suppressed to zero. Once sets of candidate peaks are identified in the accumulator, start and 
end points of line segmentation associated with those peaks are identified next. If two line 
segments associated with the each other but separated by less then predefined gap 
threshold, are merge into a single line. Furthermore the lines that have both Hough 
parameters within the predefined threshold also merge in order to avoid multiple lines on 
same location. The start and the end points of line segments represent the outline of the 
underwater pipeline. 

Due to noise and underwater conditions parts of the object boundary are detected. To draw 
a full boundary of the pipeline over an image a slight different approach is adopted. The 
first and last points of the line segment have been used to calculate the full boundary of the 
object using line equation. Once the slope of the line is computed from the line equation a 
Bresenham line algorithm, which is one of the oldest algorithms in computer graphics is 
used to construct a noise free boundary of the object. Bresenham line algorithm have few 
advantages, first it is relatively faster and simple to implement and it is robust if part of the 
pipeline is not visible or occluded. Fig. 4 shows the result of Hough transform and 
Bresenham line algorithm on underwater image. 
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Fig. 4. Results of a) Edge image, b) Line segments detection using Hough Transform and 
c) Final image using Bresenham line Algorithm. 



3. Underwater pipeline Model 

After detecting the underwater pipeline in an image, the next phase is the design of 
deformable template that represents the perspective view of underwater pipeline 
boundaries. The deformable template uses a prior shape model that can be seen as a 
regularization term in the fitting process. To model the underwater pipeline in this work, 
second order non-uniform B-spline curve with six control points is used. The first three 
control points use to define the left boundary, while the last three control points are use to 
define right boundary of the pipeline as shown in Fig 5. The interval of the B-spline function 
is [0, 2] and the knot multiplicity on intervals are ko=ki=k2=0, k3=k4=k5=k6=k7=k8=l and 
k9=kio=kn=2. The boundary contour c(s) = (x(s), y(s)) is then represented using a B-spline 
function is given below: 

x(s) = V 5 _ BjOXJ, 0<s<2 (5) 

where Q= ^ q * ^ ^ ^ q ^ , and b(s) = (B (s),..., b 5 (s)) and similarly for y(s). The contour c(s) 
of the pipeline boundary is also represented by a vector Q with the B-spline basis U(s), so that: 

c(s) = U(s)Q for < s < 2 
where 

and q = (q* q> y . The h denotes the 2x2 matrix, ® is the Kronecker product and Q is the x-y 

coordinate of the B-spline curve. The matrix representation of the computed spline function 
is given in (7) and (8). 

*iO) = (l s s 2 ) M [q x _ x q x Q q\ q\ q* q x 4 ) ( 7 ) 

x 2 0) = (l s s 2 ) M l [q x _ x ql q{ q\ q\ q x 4 ) (8) 



and similarly for y(s). Mo and Mi are the span zero and span one matrix respectively as 
shown below: 
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The developed B-spline function provides convenient framework to track underwater 
pipeline in an image sequence over simple polynomial function or line function. The main 
disadvantage of the line or polynomial function is no local control. Since the features of 
underwater pipeline are random and small variation in any edge location will case big 
change in the orientation of line function or in polynomial function. Moreover these 
functions are not suitable to track flexible underwater installations, such as fiber optic 
cables. The main advantage of the designed underwater pipeline model is the local control 
because variation in features location only effect on a part of curve instead of whole curve 
and can track rigid and non-rigid installation without any modification. 
The B-spline model used in this project has six control points. These six control points give 
12 degree of freedom. It allows the arbitrary deformation of the contour, which does not 
happen for any real object. It is desirable to restrict the displacement of this control points to 
a lower dimensional space. It is assumed that the variation of pipeline boundaries in an 
image is linear and described by a shape space planar affine transformation. Affine shape 
space has 6 degrees of freedom, gives perspective effects and can handle translation and 
rotation. Affine shape space can be viewed as the class of all linear transformation that can 
be applied to a template curve Co(s) as in (9) 




Fig. 5. B-spline contour that represents left and right boundaries of underwater pipeline, 
posted on underwater pipeline image. 



c(s) = u + Mc (s) 



(9) 



where u is a two dimensional translation vector, and M is a 2x2 matrix which corresponds to 
remaining four affine motion. The affine space can be represented in a shape space (Blake & 
Israd 1998) with template Qo and shape vector X. A shape space is a linear mapping of a 
" shape-space vector" X to a spline vector Q as in (10) 



Q = ^x + Qo 



(10) 



where W is Nq by Nx shape matrix, X is a shape vector X G S, S is the set of all possible 
configuration of state and Nq and Nx are the dimensions of spline vector and shape vector 
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respectively. X also called state vector because it represent the current state of the underwater 
object and Qo is a template curve. The matrix Wand shape space vector X are described as: 



w-- 



10QS QJ 
1 Ql QS 



X = [d l d 2 A n -\ A 22 -l A 2l A u ] 

The first two column of the shape matrix W represents the two dimensional (2D) translation and 
the remaining four columns comprise one rotation and three deformations (horizontal, vertical 
and diagonal). The dimension of the shape space N x is usually small compared to the size of the 
spline vector Nq. However the underwater pipeline or cable tracking required only 2 degree of 
freedom such as translation and rotation of the pipeline with respect to the center of the image 
plane and it is necessary to further reduce the degree of freedom of shape space. The further 
reduction of the shape space is achieved by using the Principle component analysis (PCA) which 
is commonly used to construct a shape space that is larger then necessary (Cootes, et al., 1995). In 
order to model the shape variation in a lower dimension L2-norm PCA (Blake & Israd 1998) is 
used. L2-norm PCA works satisfactorily compare to classical PCA as a mean of spline based curve 
representation. To measure the shape variations, various characteristic poses of underwater 
pipeline are taken as a training sets {Xk, k=l, . . V M}. With the help of these training sets a mean and 
the covariance matrix are calculated using the (11) and (12). 

x^ifx, (11) 

and 

P=^I(x,-x)(x,-xy ( 12 ) 

After that eigenvector and eigenvalues are computed with the multiple of covariance 
matrix P and the spare of the B-spline function H as shown in (13). This spare provides the 
invariance of the re-parameterization which is very important factor for measuring the 
difference between two spline based curve. 

H= ^ih )B{s)Tds (13) 

Once eigenvectors are computed in descending order of eigenvalues, W has been 
constructed next. 

W" = (v x ,-,v N , x ) (14) 

where v represents eigenvectors. The parameters Qq and W of the shape subspace are 
given below: 

Q'o=^X + Q (I 5 ) 

W' = WW" (16) 

This compression reduced the shape space significantly, which is very important for visual servo 
application. Further, it also reduced the processing time as the matrix dimensional also reduced. 
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4. Feature extraction and curve fitting 

Given an image containing the target, the measurement process consists of casting normals 
ft(s) (called measurement line) at pre-specified points around the initial or current estimated 
contour as shown in Fig 6a. To extract the feature curve in the image, one dimensional feature 
detector is applied along each measurement line. The feature detector is simply a scanner that 
scans for intensity variation on the binary image obtained after Hough transform and Bresenham 
line algorithm. The measurement lines are unit normal vectors and the slopes of these normals are 
computed by differentiating the span zero B-spline function as shown in (17). 



x' l (s) = (0 1 2s) M [q x _ x q* q* q\ q\ 



and 



y[(s) = {0 1 2.) M {q\ 



% 



% Vi 



(17a) 



(17b) 



and similarly for span one B-spline function. The distance from a feature to the contour is called 
the innovation of the feature and is given in (22). The images obtained after Hough transform and 
Bresenham line algorithm ideally gives one feature point (left and right boundary of the pipeline 
or cable) along the normal as shown in Fig 6b. However due to sporadic marine growth and 
dynamic underwater conditions, there may be no or more than one feature along the normal. It 
required some mechanism for evaluating the measurement and picking the correct feature point 
that used in subsequent calculations. This evaluation is done by using the innovation of the feature 
which is the distance between the estimated curve point and the feature curve point. If there are 
more features, the innovation that gives the minimum distance is selected and if the selected 
innovation is greater than the predefine value, the measurement is invalidated. 




S I0D 150 1 30 3D H « 1D0 16B 2ffl 

(a) (b) 

Fig. 6a) Measurement line on B-spline curve for feature extraction b) Dots show the feature 
extracted using the one-dimensional feature detector. 

After extracting the feature points in an image, the next part of tracking algorithm is to use 
curve fitting technique to measure the current position and orientation of the underwater 
pipeline. If Cf(s) expressed the image feature curve obtained using the one dimensional 
feature detector and co(s) is a pattern curve than, the whole tracking is the estimate c(s), a B- 
spline curve that is a deformation of co(s) and that approximate c/(s). This approximation can 
be express as a minimization problem: 
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r - argmin lc(s) - C/(X) (17) 

which is the square of the residual norm. Generally, measurements made from images are 
noisy due to dynamic nature of underwater environments and several other reasons. It is 
necessary to increase the tolerance for image noise. To overcome the effect of noise a mean 
contour shape and Tikhonov regularization are used to bias the fitted curve toward the 
mean shape c m to the degree determined by regularization constant as shown in (18). 

r = argminf Q 2 \\c(s) - c m (s)f + jc(s) - c f (s)f j ( 18 ) 

where c m (s) is the mean shape and a is the regularization parameter. If the regularization 
parameter is very large, the term |U(j) — cy(^)| is negligible to that of Q 2 |c(»-c w O)| in (18). 

With a large amount of regularization, the data and any noise on the data can be ignored 
effectively. On the other hand if Q is small, the weighting placed on the solution semi norm 
is small and the value of the misfit at the solution become more important. Of course, if Q is 
reduced to zero, the problem reduces to the least-square case as in (17), with it extreme 
sensitivity to noise on the data. Equation (19) shows the fitting equation in term of shape 
state vector X. 

min X = Q 2 ||X - xf + ||Q - Q, f with Q = WX + Q (19) 

To avoid the influence of position and orientation of the mean contour and from the features of 
other objects in the background in the regularization term, weight matrix L s is introduced as in (20). 

min X = ||X - x| r L s ||x - x|| + ||Q - Q f f (20) 

where L s =QH and H is the spare of B-spline function as defined in (13). Since actual 
image processing is discrete, by using the definition given in (Blake & Israd 1998) the curve 
fitting problem is expressed in a discrete form as follows: 

minX IX-xfrlX-Xl + X^^-h^O^X-X]) ( 21 ) 

where V{ and h(si) T are given in (22) and (23) respectively. Introducing the concept of 
information matrix S; and information weight sum Zi from the stochastic process, the 
algorithm for finding the best-fitting curve is summarized as follows: 

• Select N regularly equal-spaced sample points 5=81, 1=1,.. .,N, with inter-sample space 
h, along the entire curve c(s) so that, in the case of an open curve si=0, Si+i=Si+h and 
sn=L. 

• For each i, find the position of Cf(s) by applying ID feature detector along the normal 
line passing though c(s) at s=Si. 

• Initialize Z = 0, S =0 
Iterate, for i=l /m . ,,N 

v,- = (cf (s t ) - c(s t )).n( Si ) (22) 



h(^) r =n(^) r ^-)^ (23) 



if I vi I <k then 
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(24) 
(25) 
Else 



where n(si) is the normal unit vector of curve c(s) at s=Si, a i = N B and k is the length 
of the measurement line. 

• The aggregated observation vector is Z=Zn with the associated statistical information 
S=Sn. 

• The best-fitting curve is given in shape-space by: 

x = x+(s + sy l z (26) 

The term | vi | <k is used for measurement validation as described earlier. Si (information 
matrix) is a measurement of the weight of each intermediate estimate X, Zi (information 
weight sum) accumulates the influence of the mean shape c m . The proof of correctness of the 
curve fitting algorithm can be found in (Blake & Israd 1998). 

5. Pose and orientation measurement 

After the shape vector, that represents the current state of underwater pipeline, is estimated, 
the goal is to guide the autonomous underwater vehicle in autonomously following the 
pipeline. It required some mechanism to relate the position and orientation of the 
underwater pipeline to the AUV. This mechanism is necessary for the AUV to keep the 
underwater pipeline within the field of view. The mean shape c m of the underwater pipeline 
introduced in (18) is used as a reference location for the underwater pipeline in an image. 
The reference position of underwater pipeline in an image can be measure by averaging the 
control points of the mean shape as in (27). 

A x =mean(Q x J (27) 

and A y =mean(Q y J 

This average actually presents the centroid of the control points which lies almost in the 
center of the image. This information is needed so that the AUV can effectively maintain the 
underwater pipeline in the center of the image acquired. The B-spline model developed to 
represent underwater pipeline in an image, is set fixed in y-direction so, the translation in y- 
direction is always zero and not required. The shape vector estimated in (26) is used to 
measure the current position of the control points using (10). The average of these control 
points gives the current position of underwater pipeline in an image as shown in (28) 

A x =mean(Q x ) (28) 

The current distance of the pipeline in an image with respect to the reference can be measure 
by subtracting the reference position from the current position as shown in (29) 

T X =A X -A X (29) 
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It is obvious that the positive value of Tx refer the pipeline as is translated to the right, and the 
negative means that the pipeline is translating toward the left side. The value of Tx can then be 
used to generate a navigational command, in order to align an AUV over the pipeline, and keep 
the pipeline in the optimum filed of view. The location of the pipeline can be use to track an 
underwater pipeline in an image, however this is an ineffective method. It may possible that the 
AUV may be facing the wrong direction after aligning, and the tracking system must calculate the 
orientation of the pipeline for correct navigation. Once the autonomous underwater vehicle has 
aligned to the location of the pipeline, the tracking system will then instruct AUV to rotate itself in 
the calculated pipeline orientation and commence autonomous tracking. 

Similar to the location measurement, at first the reference orientation of underwater pipeline 
is measured using the mean curve control points. As mentioned earlier, the perspective 
view of underwater pipeline is modeled using the left and right B-spline curve. The 
orientation of the left and right reference boundaries are given below: 

. tan - i'*LZ*Ll . 5 - tan-- {<Z£\ . ^d R = m - 



?o-?i 



R _ tan -M^Z^V 5 - tan ->K-n and = ,(£z£ 



and 



q 2 -q,j yq A -q^j \<12-<1* 

where r r r r r and r are the angles between the left and right boundaries 

control points. These angles now use as reference angles to measure the pipeline variation. 
The orientation of the current estimated control points are measured in a similar fashion 

*„ -tan" 1 ! fail' * ^tan-'fctf]' ™ d R„=M* 



and 

^=tan-{4^} * S2 =ta„-'f4^} and R Ri =uA^ 

After measuring the orientation of each current estimated control points, the Euclidean 
angle between the two corresponding control points are measured as shown below: 

R =R -R / R =R -R and,/? =r -r 

ly \ IV L\ 1V L\' 1V 2 1V L2 1V L2 ' iV 3 iV Z,3 iV Z,3 

R =R -R r R =R -R anc l R = R -R 

Once all the Euclidean angles have measured, the minimum Euclidean angle is selected as a 
rotation angle for the AUV. 

R = mm(R l ,R 2 ,R 3 ,R 4 ,R 5 , and R 6 ) 

After the translation and rotation of the underwater pipeline with respect to the mean curve 
are measured, the autonomous underwater vehicle has aligned and orientated itself in the 
direction of the pipeline. The AUV subsequently start moving in the direction of the 
underwater pipeline and begin autonomous navigation. 
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6. Dynamic tracking 

Any tracking system required a model of how the system is expected to evolve or behave 
over time (MacCormick, 2000). In this work, second order auto-regressive process or ARP is 
used. An autoregressive process is a time series modeling strategy which takes into account 
the historical data to predict the current state value. The simplest autoregressive model is 
the linear model where the AUV is assumed to have a constant velocity model with respect 
to the object. It is best described by the following second order autoregressive model: 



X, - X = A 2 (X t _ 2 - X) + A x (X,_! - X) + B w k 



(26) 



where w is a random Gaussian noise with zero mean and unit standard deviation, X is the 
steady state mean and X t is the position of object at time t. The matrices A and B are 
representing the deterministic and stochastic parameters respectively. These parameters are 
needed to be tuned appropriately for expected motion in order to obtain best tracking results. 
If P and / are expressed the damping rate and the frequency of oscillation of the harmonic 
motion respectively then according to the theory of control system they must set to zero for 
constant velocity model, so that the coefficients of the dynamic model are defined as: 

(27) 
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where A\ and A2 are standard for all second order constant velocity model. The problem is 
the estimation of Bo and it required a tuning from the experiment because it defines the 
standard deviation of the noise. Equation 26 can be simplified by defining: 



and then (26) can be rewritten as: 
where 



Zt-Z = A (Zt-i-Z) + B *k 



(28) 



(29) 



The second order state % t has a mean and covariance is given below: 

Zt=£[Zt] and P t =v\Z t ] 

A Kalman filter is design to merge the information from the predicted state and the best 
fitting curve obtain from (26). A complete one step cycle of tracking is given below: 
1. Predict shape space vector % t using the dynamic model: 



Zt-Z = AZt-\-Z) 

P t = AP t _ x A T + BB T 

2. Apply (22) to (26) to estimated best fitted state of object. 

3. For each measurement the state estimation is update as follows: 



(30) 
(31) 
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and 



: t =p t H T (s t np t n T + n 

Z t =Z t +K t Z t 

P t =(l-K t S t B)P t 

H = (0 /) 



(32) 

(33) 
(34) 

(35) 



The term Z t and St are aggregated observation vector and associated statistical information 
respectively. If the measurement failed along the normal due to occlusion or multiple features so 
that Z t = and St = 0, and the state of underwater pipeline is estimated without modification. 



7. Results and Discussion 




e) f) 

Fig. 7a Results of Perona-Malik filter with N=10 a) original image b) K = 5 c) K = 10 d) K = 
15 e) K = 25 f) K = 40. 

This section presents the results which were obtained by testing the proposed underwater 
pipeline tracking system using AUV real image sequences. As a first step of feature 
extraction, image filtering was performed to reduce the noise in the image sequence. The 
key parameters in the Perona-Malik filter are the threshold level for noise removing (K) and 
the number of iterations N. The noise threshold level (K) was analyzed with different values 
of K's with fix number of iterations. Fig. 7a and Fig. 7b show the effect of Perona Malik filter 
with varying values of K on synthetic and real images respectively. It is observed that the 
increasing the value of K also increase the smoothness of image while preserving the object 
boundary. The best results were obtained at K=15. Large value of K effectively reduces the 
noise in an image but they also smooth the object boundary and edge detection may fail to 
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detect. Based on these visual analysis the value K=15 was selected for further filtering 
process. The second important parameter in Perona Malik filter is the number of iterations. 
Much improved results can be obtained using the large number of iterations however, it also 
increase the computational time for filtering which is not at all required for robotic 
application. To achieve the optimal performance the number of iterations are fixed to N=10 
only. Fig. 8 shows the comparison of edge detection results between the original and filtered 
underwater image sequences. The average number of edge filtering is approximately 770 
pixels with K=15 and N=10 iterations. These reductions of edges significantly improve the 
results of feature detection. 




~~* M '-i E a - " ™^- 

e) f) 

Fig. 7b. Results of Perona-Malik filter with N=10 a) original image b) K = 5 c) K = 10 d) K 
15 e) K= 25 f) K = 40. 
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Fig. 8. Edge detection comparisons between original and filtered underwater image sequence. 
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Fig. 9. Results of Hough transform and Bresenham line algorithm. First column original image, 2 nd 
column results of Hough transform, and 3 rd column is the results of Bresenham line algorithm. 



Underwater pipeline detection 


Performance 


% 


Correctly detected 


419 


93.11% 


Not detected 


31 


6.88% 



Table 1. Performance measurement for the underwater pipeline detection. 

After filtering the image, the next part of the image processing involves the detection of the 
underwater pipeline in an image sequences using Hough transform. In order to perform the 
pipeline detection, the 6 parameter was quantized into 180 levels and the p parameter was 
quantized using the root mean square value of total numbers of row and column of image 
frame. The gap threshold was set to 25 levels and the minimum segment length was 30 
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pixels. These values were obtained by a series of experiments on real underwater images. 
After detecting the pipeline segments in an image, Bresenham line used to draw the pipeline 
boundary. Fig. 9 shows the results of Hough transform and the Bresenham line algorithm on 
real underwater images. The Hough transform and Bresenham line algorithm successfully 
detect and draw the boundary of underwater pipeline. Table 1 displays some performance 
results calculated using a batch of 450 real images. 

These results can be further improved upon, if the Hough parameter space is increased. 
Additionally canny edge detection can also be use to improve the accuracy of edge mapping. 
Although these amendments increase the probability of underwater pipeline detection, it 
may results in an increment of false detections and also increases computational time. 
After detecting the pipeline in an image frame the next phase was the extraction of feature points 
from an image. In general, the accuracy and the performance of the tracking algorithm improve as 
the number of feature points, used in curve fitting stage increases. However, as the number of 
feature increases the computational load become heavier. There is an obvious trade-off between 
accuracy of the tracking algorithm and the computational time. To achieve the balance between 
performance and efficiency, 20 feature points were used. It was observed from the 
experimentation that, at least five feature points on either side are required for successful 
deformation of template contour to the image features. If there are fewer features then the state of 
the underwater pipeline was predicted without modification. Fig 10 depicted the underwater 
pipeline tracking results obtained using the Kalman tracking algorithm. Every 10 th frame of 4500 
frames sequences was used to check the robustness and the accuracy of the tracking algorithm. 
The position and the orientation of the underwater pipeline boundary computed are summarized 
in the table 1. Fig 11 illustrates the predicted, updated and the actual position of the underwater 
pipeline. It is observed that the maximum absolute error in the translation is around 12 pixels 
which show the robustness of tracking algorithm. To solve the initial value problem of the Kalman 
filter, it has been assumed that when tracking is started pipeline is near the center of the image. In 
other words, mean shape X and covariance P used as the initial value for Xto=X. 





Actual Position 


Estimated Position 


Error 


Estimated Angle 


Average 


197.38 


197.98 


-0.60 


0.34 


Max 


224.33 


217.45 


10.75 


2.80 


Min 


179.54 


171.30 


-12.65 


-0.99 


Mode 


193.77 


- 


3.46 


- 



Table 2. Summary of Kalman Tracking Results. 



8. Conclusion 

In this paper a robust vision based system for underwater pipeline tracking has been 
presented. The developed system successfully detects the pipeline and track in real image 
sequences. The algorithm has been implemented in Matlab environment and all test have 
been conducted on a 1.70 GHz Pentium IV machine executing Windows XP. The resolution 
of both synthetic and real image sequences is 200 x 370 pixels. 

The B-spline contour deforms successfully, based on the features extracted with the series of 
image processing technique and the orientation and the position of the pipeline has been 
computed. The Kalman filter has been used to track the pipeline boundary in an image 
sequences. The system efficiently track the pipeline when it is fully or partially covered by 
the sand or marine flora and even in clustering situations. 
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To conform the validity of the purposed system many experiments conducted on real 
and synthetic underwater pipeline image sequences. The maximum error that has been 
achieved is less then 10 pixels which show the robustness of the purposed system. Since 
the purposed system implemented in Matlab so, it takes 1 sec to process each frame. In 
order to improve the processing time the system will be implemented in C++. 
Furthermore in the case of multiple pipelines and to achieve better performance particle 
filtering technique will be explored. 




tbULfc 




Fig. 10 Underwater pipeline tracking results with Kalman filter. 
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Fig. 11. Comparison of actual and predicted and updated position of underwater pipeline 
using the Kalman tracking algorithm. 
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1. Introduction 

Most of underwater pipeline tracing operations are performed by remote operated vehicles 
(ROVs) driven by human operators. These tasks often require continued attention and 
knowledge/ experience of human operators to maneuver the robot (Foresti. G.L. and 
Gentili.,2000). In these operations, human operators does not require an exact measurement 
from the visual feedback, but based on the reasoning. 

For these reasons, it is desirable to develop robotics vision system with the ability to mimic 
the human mind (human expert's judgement of the terrain traverse) as a translation of 
human solution. In this way, human operators can be reasonably confident that decisions 
made by the navigation system are reliable to ensure safety and mission completion. To 
achieve such confidence, the system can be trained by expert (Howard. A. et al, 2001). 
In order to enable robots to make autonomous decision that guide them through the most 
traversable regions of the terrain, fuzzy logic techniques can be developed for classifying 
traverse using computer vision-based reasoning. Computing with words is highly 
recommended either when the available information is too imprecise to use numbers, or 
when there is a tolerance for imprecision which can be exploited to get tractability and a 
suitable interface with the real world (Zadeh. L, 1999). 

Current position based navigation techniques cannot be used in object tracking because the 
measurement of the position of the interested object is impossible due to its unknown 
behavior (Yang Fan and Balasuriya, A, 2000). The current methods available in realizing 
target tracking and navigation of an AUV, used optical, acoustic and laser sensors. These 
methods have some problems, mainly, in terms of complicated processing requirement and 
hardware space limitation on AUVs (Yang Fan and Balasuriya, A, 2000). Other relevant 
research consists of neural-network terrain-based classifier in Foresti. et.al (2000) and 
Foresti, G. L. and Gentili (2002). Also, existing method using Hough transform and Kalman 
filtering for image enhancement has also been very popular (Tascini, G. et al, 1996), 
(Crovatot, D. et al, 2000), (El-Hawary, F. and Yuyang, Jing, 1993) , (Fairweather, A. J. R. et al, 
1997) and (El-Hawary, F. and Yuyang, Jing, 1995). 

2. Research Approach 

Visible features of underwater structure enable humans to distinguish underwater pipeline 
from seabed, and to see individual parts of pipeline. A machine vision and image processing 
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system capable of extracting and classifying these features is used to initiate target tracking 
and navigation of an AUV. 

The aim of this research is to develop a novel robotics vision system at conceptual level, in 
order to assist AUV's interpretation of underwater oceanic scenes for the purpose of object 
tracking and intelligent navigation. Underwater images captured containing object of 
interest (Pipeline), simulated seabed, water and other unwanted noises. Image processing 
techniques i.e. morphological filtering, noise removal, edge detection, etc, are performed on 
the images in order to extract subjective uncertainties of the object of interest. Subjective 
uncertainties became multiple input of a fuzzy inference system. Fuzzy rules and 
membership function is determined in this project. The fuzzy output is a crisp value of the 
direction for navigation or decision on the control action. 

2.1 Image processing operations 

For this vision system, image analysis is conducted to extract high-level information for 
computer analysis and manipulation. This high-level information is actually the 
morphological parameter for the input of a fuzzy inferences system (linguistic 
representation of terrain features). 

When an RGB image is loaded, it is converted into gray scale image. RGB image as shown in 
Fig. 1. Gray-level thresholding is then performed to extract region of interest (ROI) from the 
background. The intensity levels of the object of interest are identified. The binary image 
B[i,j], is obtained using object of interest's intensity values in the range of [Ti, T2] for the 
original gray image F[i,j]. That is, 

B[i,n4 ww* (i) 

[0 otherwise 

The thresholding process producing a binary image with a large region of connected pixels 
(object of interest) and large amount of small region of connected pixels (noise). Each region 
is labeled and the largest connected region is identified as object of interest. In the labeling 
process, the connected pixels are labeled as either object of interest or unwanted objects by 
examining their connectivity's (eight-connectivity) to neighboring pixels. Label will be 
assigned to the largest connected region that represents the object of interest. 
At this stage, feature extraction is considered completed. The object of interest is a pipeline 
laid along the perspective view of the camera. The image is segmented into five segments 
and processed separately for terrain features as multiple steps of inputs for the fuzzy 
controller. In order to investigate more closely each specific area within the image segment, 
each segment is further divided into six predefined sub segments in the image. Each sub 
segment (as illustrated by Fig. 2) is defined as follows. 

• Sub segment 1 = Upper left segment of the image 

• Sub segment 2 = Upper right segment of the image 

• Sub segment 3 = Lower left segment of the image 

• Sub segment 4 = Lower right segment of the image 

• Sub segment 5 = Upper segment of the image 

• Sub segment 6 = Lower segment of the image. 

A mask image with constant intensity is then laid on the image as shown in Fig. 3. This is 
actually an image addition process whereby it will produce a lighter (highest intensity 
value) area when intersects the region of interest. The remaining region with highest 
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intensity value then be calculated its coverage area in the image as shown in Fig. 4. The area, 
A of the image is determined by. 



n m 






(2) 



Sub segment 5-6 are being determined its location relative to the image center. Coverage 
area and location of object of interest in each sub segment is finally be accumulated as 
multiple input of the fuzzy inference system. 



Fig. 1. Typical input image (RGB) 



Fig. 2. Show image sub segment 




Fig. 3. Mask on threshold, removed noise image. 
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Fig. 4. Acquired area information. 



2.2 The fuzzy inference system 

The fuzzy controller is designed to automate how a human expert, who is successful at this 
task, would control the system. The multiple inputs to the controller are variables defining 
the state of the camera with respect to the pipeline, and the single output is the steering 
command set point. Consider the situation illustrated by Fig. 5. The fuzzy logic is used to 
interpret this heuristic in order to generate the steering command set point. In this case, the 
set point of AUV has a certain amount (AX) to the right. 

Basically, a human operator does not require a crisp / accurate visual input for mission 
completion. There are total of six inputs based on the image processing algorithm. 

• Input variable 1, xi = Pipeline area at upper left segment in the image 
Input variable 1 fuzzy term set, T(xi) = {Small, Medium, Large} 
Input variable 1 universe of discourse, Ufa) = [0.1 -1.0] 

• Input variable 2, xi = Pipeline area at upper right segment in the image 
Input variable 2 fuzzy term set, Tfa) = {Small, Medium, Large} 
Input variable 2 universe of discourse, Ufa) = [0.1 -1.0] 

• Input variable 3, X3 = Pipeline area at lower left segment in the image 
Input variable 3 fuzzy term set, Tfa) = {Small, Medium, Large} 

Input variable 3 universe of discourse, Ufa) = [0.1 -1.0] 

• Input variable 4, X4 = Pipeline area at lower right segment in the image 
Input variable 4 fuzzy term set, Tfa) = {Small, Medium, Large} 
Input variable 4 universe of discourse, Ufa) = [0.1 -1.0] 

• Input variable 5, xs = End point of pipeline relative to image center point 
Input variable 5 fuzzy term set, Tfa) = {Left, Center, Right} 

Input variable 5 universe of discourse, Ufa) = [0.1 -1.0] 

• Input variable 6, xe = Beginning point of pipeline relative to image center point 
Input variable 6 fuzzy term set, Tfa) = {Left, Center, Right} 

Input variable 6 universe of discourse, Ufa) = [0.1 -1.0] 
The only fuzzy output. 

• Output variable 1, yi = AUV steering command set point 

Output variable 1 fuzzy term set, T{yi) = {Turn left, Go straight, Turn right} 
Output variable 1 universe of discourse, V(yi) = [0 -180] 
The input vector, x is. 
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The output vector, y is. 



X= (Xi, X 2/ X3, X4, x 5/ x 6 ) T 



y = (yi) J 



(3) 



(4) 



Gaussian and n-shaped membership functions are selected in this case to map the input to 
the output. Gaussian curves depend on two parameters o and c and are represented by. 



f(x;cr,c) = exp 



~(x-c) 
la 2 



(5) 



n-shaped membership function are represented by. 

r/ , . \S(x;c — b,c — b/2,c) for x < c 
f(x;b,c) = < 

[1- s(x;c,c + b 1 2,c + b) for x > c 

where S(x; a, b, c) represents a membership function defined as. 

for x < a 



(6) 



S(x;a,b,c) = 



2{x — a) 



(c - a) 

2 
2(x-a) 

l_ 2 

(c- a) 



for a < x < b 



for b < x < c 



(7) 



for x > c 



In the above equation, a, a, b and c are the parameters that are adjusted to fit the desired 
membership data. Typical input variable and output variable membership function plot are 
shown in Fig. 6 and Fig. 7. 
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There are totally 13 fuzzy control rules. The rule base as shown in Fig. 8. 



Auvelirectsv = 60.6 



L^ 



L^l 



Fig. 8 Rule viewer for fuzzy controller. 

In order to obtain a crisp output, the output fuzzy set is then aggregated and fed into a 
centroid (center of gravity) method defuzzification process. The defuzzifier determines the 
actual actuating signal, y y as follows. 



13 



(8) 
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3. Simulation and Experimental Results 

The simulation procedures are as follows: 

a. Define the envelope curve (working area) of prototype. 

b. Give the real position and orientation of pipeline defined on a grid of coordinates. 

c. Predefine the AUV drift tolerance limit (±8.0cm) away from the actual pipeline 
location. 

d. Initiate the algorithm. 

e. AUV navigating paths are recorded and visualized graphically. 

The algorithm has been tested on computer and prototype simulations. For comparative pur 
poses, the results before and after fuzzy tuning are presented. Typical examples of results 
before fuzzy tuning are shown in Fig. 9 and Table 1. 




Fig. 9. AUV path (no proper tuning). 



AUV path 


Actual location 
x-axis (cm) 


Simulated 

result x-axis 

(cm) 


Drift (cm) 


Percentage of 
Drift (%) 


5 


47.5 


69.5 


+22.0 


275.0 


4 


58.5 


71.7 


+13.2 


165.0 


3 


69.6 


73.3 


+3.7 


46.3 


2 


80.8 


78.3 


-2.5 


31.3 


1 


91.9 


75.7 


-16.2 


202.5 



Table 1. Data recorded (without proper tuning) 



400 



Mobile Robots, Towards New Applications 



Typical examples of results after fuzzy tuning are shown in Fig. 10 and Table 2. 
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Fig. 10. AUV path (with proper tuning). 



AUV path 


Actual location 
x-axis (cm) 


Simulated 

result x-axis 

(cm) 


Drift (cm) 


Percentage of 
Drift (%) 


5 


47.5 


55.2 


+7.7 


96.3 


4 


58.5 


57.4 


-1.1 


13.8 


3 


69.6 


68.5 


-1.1 


13.8 


2 


80.8 


88.1 


+7.3 


91.3 


1 


91.9 


85.9 


-6.0 


75.0 



Table 2. Data recorded (with proper tuning). 

The simulation results show that the drift within tolerance limit is achievable when proper 
tuning (training) is applied to the fuzzy system. The percentage of drift is considered 
acceptable , as long as it is less than 100%, since this implies the path is within the boundary. 
The effectiveness of the system has been further demonstrated with different target 
orientation and lighting conditions. 



4. Conclusions 

This paper introduces a new technique for AUV target tracking and navigation. The image 
processing algorithm developed is capable of extracting qualitative information of the terrain 
required by human operators to maneuver ROV for pipeline tracking. It is interesting to note 
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that fuzzy control system developed is able to mimic human operators 7 inherent ability for 
deciding on acceptable control actions. This has been verified experimentally and the result is 
favourable, i.e. within 8.0 cm of drift tolerance limit in a 1.5m x 2.0m working envelope. One of 
the most interesting parts being the system ability to perform target tracking and navigation 
from the knowledge of interpreting image grabbed in perspective view from the terrain. 
It should also be noted that the system offer another human-like method of representing 
human experience and knowledge of operating a ROV, rather than being expressed in 
differential equations in the common PID-controller. Obviously, the system does not require 
sophisticated image processing algorithm such as Kalman filtering or Hough transform 
techniques. All input variable required are merely an approximate value for mission 
completion, just like a human vision system. The simplicity of the system is further 
recognized when a priori knowledge of the terrain is not necessary as part of the algorithm. 
Currently a priori knowledge is required by some of the available pipeline tracking 
techniques such as (Evans, J, et al, 2003) and (Arjuna Balasuriya and Ura, T, 2002). The 
processing time is therefore reduced. 

In general the whole computational process for this prototype is complex and it usually 
takes about 60 seconds to arrive at its desired output for 5 steps (22.5cm for each step), 
which is not practical for commercial standard requirement that is at least 4 knot (2m/ s) of 
AUV speed. Commercial standard requirement of a survey AUV can be found in (Bingham, 
D. ,2002). However, the proposed system would be a workable concept for its capability to 
look forward and perceive the terrain from perspective view. As illustrated in Fig. 11, the 
perceived conditions from the second image captured could be processed concurrently 
while the AUV completing the forth and fifth step based on the previous image information. 
This will improve the processing time to support high speed AUV application. 
In addition, further studies on improving the program structure and calculation steps may help 
to achieve better computation time. Future development of transputer for parallel processing or 
higher speed processor can also be expected to bring the system into practical use. 



250- 



T 



200!? 



100 - 



2nd image blind area 







Capture 
2nd image 






50 






1 st image blind area 



Capture 
1 st image 



50 100 

Fig. 11. AUV path and its image capturing procedure. 



'5C 



402 Mobile Robots, Towards New Applications 

5. References 

Arjuna Balasuriya & Ura, T. (2002). Vision-based underwater cable detection and following 

using AUVs, Oceans ' 02 MTS/IEEE, 29-31 Oct. 2002, Vol. 3, pp. 1582-1587, 2002 
Bingham, D. , Drake, T. , Hill, A. , & Lott, R. (2002). The Application of Autonomous 

Underwater Vehicle (AUV) Technology in the Oil Industry - Vision and Experiences, 

FIG XXII International Congress Washington, D.C. USA, April 19-26 2002. 
Crovatot, D. , Rost, B. , Filippini, M. , Zampatot, M. & Frezza, R. (2000). Segmentation of 

underwater Images for AUV navigation, Proceedings of the 2000 IEEE international 

conference on control applications, 25-27 September 2000, pp. 566-569, 2000 
El-Hawary, F. & Yuyang, Jing. (1993). A robust pre-filtering approach to EKF underwater 

target tracking, Proceedings of OCEANS '93. Engineering in Harmony with Ocean, 18- 

21 Oct. 1993, Vol.2, pp. 235-240, 1993. 
El-Hawary, F. & Yuyang, Jing. (1995). Robust regression-based EKF for tracking underwater 

targets, IEEE Journal of Oceanic Engineering, Vol.20, Issue.l, pp.31-41, Jan. 1995. 
Evans, J.; Petillot, Y.; Redmond, P.; Wilson, M. & Lane, D. (2003). AUTOTRACKER: AUV 

embedded control architecture for autonomous pipeline and cable tracking, OCEANS 

2003. Proceedings, 22-26 Sept. 2003, Vol 5, pp. 2651-2658, 2003. 
Fairweather, A. J. R. , Hodgetts, M. A. & Greig, A. R. (1997). Robust scene interpretation of 

underwater image sequences, IPA97 Conference, 15-17 July 1997, Conference 

Publication No. 443, pp. 660-664, 1997. 
Foresti. G.L. & Gentili. S. (2000). A vision based system for object detection in underwater 

images, International journal of pattern recognition and artificial intelligence, vol. 14, no. 2, 

pp. 167-188, 2000 
Foresti, G. L. & Gentili, S. (2002). A hierarchical classification system for object recognition in 

underwater environments, IEEE Journal of Oceanic Engineering, Vol. 27, No. 1, pp 66- 

78,2002. 
Howard. A. , Tunstel. E. , Edwards. D. & Carlson. Alan. (2001). Enhancing fuzzy robot 

navigation systems by mimicking human visual perception of natural terrain 

traversability, Joint 9 th IFSA World Congress and 20 th NAFIPS International Conference, 

Vancouver, B.C., Canada, pp. 7-12, July 2001 
Tascini, G. , Zingaretti, P. , Conte, G. & Zanoli, S.M. (1996). Perception of an underwater 

structure for inspection and guidance purpose, Advanced Mobile Robot, 1996., 

Proceedings of the First Euromicro Workshop on , 9-11 Oct. 1996, pp. 24 - 28, 1996. 
Yang Fan & Balasuriya, A. (2000). Autonomous target tracking by AUVs using dynamic 

vision, Proceedings of the 2000 International Symposium on Underwater Technology, pp. 

187-192, 23-26 May 2000. 
Zadeh. L. (1999). From computing with numbers to computing with words -from 

manipulation of measurements to manipulation of perceptions, IEEE Transactions on 

Circuits and Systems, 45(1), pp. 105-119, 1999. 



20 



The Surgeon's Third Hand an Interactive 
Robotic C-Arm Fluoroscope 

Norbert Binder 1 , Christoph Bodensteiner 1 , Lars Matthaus 1 , 
Rainer Burgkart 2 and Achim Schweikard 1 

Unstitut fur Robotik und Kognitive Systeme, Universitat zu Lilbeck 
2 Klinikfur Orthopadie und Sportorthopadie, Klinikum Rechts der Isar, TU-Munchen 

Germany 



1. Introduction 

Industry is using robots for years to achieve high working precision at reasonable costs. When 
performing monotonous work, attention of human operators weakens over time, resulting in 
mistakes. This increases production costs and reduces productivity. There is also a constant 
pressure to reduce costs for industrial processes while keeping or increasing their quality. 
The idea of integrating robots into the OR was born over a decade ago. Most of these robots are 
designed for invasive tasks, i.e. they are active tools for medical treatment. Some are 
telemanipulation systems, filtering tremor and scaling the movements of the user. Others move 
according to pre-operatively calculated plans positioning instruments of all kinds. Main goal was 
to achieve a higher precision in comparison to human surgeons, often ignoring the time- and 
financial aspect. As the economic situation at hospitals becomes more and more strained, 
economic factors such as costs, time and OR-utilization become more and more important in 
medical treatment. Now, only few systems can fulfil both requirements: increase precision and 
reduce the duration of an intervention. 



Fig. 1. Robotized C-arm. 



Hi nl 

m 1 




ii 




























l< 









404 



Mobile Robots, Towards New Applications 



We want to introduce another type of robot which assists the surgeon by simplifying the handling 
of everyday OR equipment. Main goal is to integrate new features such as enhanced positioning 
modes or guided imaging while keeping the familiar means of operation and improving workflow. 
The robotic assistance system works in the background until the user wants to use the additional 
features. On base of a common non-isocentric fluoroscopic C-arm we will explain the way from a 
manually operated device into an interactive fluoroscope with enhanced positioning and imaging 
functionality. We first discuss problems of a common C-arm and present possible solutions. We 
then examine the mechanical structure and derive the direct and inverse kinematics solutions. In 
the next section, we describe how the device was equipped with motors, encoders and controllers. 
Finally, we discuss the results of the functionality study and show ways to improve the next 
generation of robotized C-arms. 



2. State of the art 

2.1 Classical Manual C-arm 

The common C-arms are small mobile X-ray units. The X-ray source (XR) and the image 
intensifier (II) are mounted on a C-shaped carriage system. Conventionally there are five 
serial axes Al to A5, two of them translational, the rest rotational: 



Axis 


Parameter 


Movement 


Name 


Function 


Al 


di 


Trans 


Lift 


Height adjustment 


A2 


e 2 


Rot 


Wig-Wag 


Rotation in the x-y plane 


A3 


d 3 


Trans 


Sliding carriage 


Arm-length adjustment 


A4 


4 


Rot 


Angulation 


C-Rotation sideways 


A5 


5 


Rot 


Orbital Movement 


C-Rotation in C-plane 



Table. 1. The axes and their names and functions. 

The joint arrangement in combination with the C-shape allows for positioning XR and II 
around a patient who is lying on the OR-couch. Besides of the lift, which is motor-driven, all 
joints have to be moved by hand one after another: the brake is released, positioning is 
performed and after reaching the target position, it has to be locked again. As shown in Fig. 
2 , the axes A4 and A5 do not intersect. In addition, A5 does not intersect the center-beam of 
the X-ray cone. Therefore, there is no mechanically fixed center of rotation (=isocenter). 




Fig. 2. A common C-arm with the axes Al to A5. 
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The overall dimensions and weight of the device are designed for mobility. The C-arm used for 
our experiments can be moved by one person and fits through normal sized doors. Thus it can be 
moved from one intervention scene such as OR or ER to the next and is not bound to one room. 

2.2 Isocentric and semiautomatic C-arms 

Meanwhile, industry paid attention to the medical requirements and developed new 
devices which can deal with the problem of the missing isocenter. The Siemens 
SIREMOBIL IsoC (Kotsianos et al. 2001, Euler et al. 2002) is constructed with a 
mechanical isocenter, i.e. A4, A5 and the center-beam intersect in the same point. The 
radius is fixed, i.e. the distance of the II to the center of rotation cannot be adjusted. As 
XR and II had to be mounted inside the C to allow for a 190° rotation of joint five, the 
size of the C had to be increased. Motorization of A5 makes it possible to perform (semi-) 
automatically data acquisition for 3D reconstruction. 

Another system which is the base of this project, is the Ziehm Vario 3D. It has no mechanical 
isocenter, but when the orbital axis is moved by the user, it can automatically compensate 
for offsets with the translational joints (Koulechov et al. 2005). This method works for the 
vertical plane only but it allows for a free selection of the radius within the mechanical 
limits. The newest version can also perform the orbital movement in a full- or semi- 
automatically motor driven way. Similar to the Siemens device, data acquisition for 3D 
reconstruction is realized this way. 

2.3 Medical problems 

When using a standard C-arm fluoroscope two problems occur: First, it is often desired to 
obtain several images from the same viewing angle during the operation. If the joints were 
moved after taking the first radiograph, the old position and orientation must be found 
again. Second, if another image from the same region but from a different angle is required, 
more than one joint must be adjusted in general. Even basic movements, such as straight- 
line (translational) movements in the image plane are difficult to perform manually, due to 
the fact that three of the joints are rotational. Similarly, pure rotations around the region of 
interest (ROI), if desired, are cumbersome, since two joints are translational, and most of the 
devices are not isocentric. This shows exemplarily the rather complex kinematics 
construction of standard C-arms. At the moment handling is done manually and without 
guidance, which often leads to wrong positioning and thus to a high number of false 
radiographs, not to forget the radiation dose. 

3. Computer Simulation 

Completing the mechanical work was judged to take some time. Therefore, a software 
simulation was created to mimic the functionality of the real device. (Gross et al. 2004). It 
allow for testing and evaluating the primary idea of the project and of a first series of 
applications (see also section 6). 

Additionally, simulated radiographs of our virtual patient can be generated. Image- and 
landmark-based positioning can be performed the same way as on the real machine now. A 
comparison of the image quality is given in Fig. 4 for a radiograph of the hip joint. Although 
the proof of concept study of the mechanical version is completed now, the simulation is 
still a valuable device for testing new concepts and ideas. Algorithms for collision detection 
and path planning are now developed on this base. 
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Fig. 3. Screenshot of the simulation GUI. 




Fig. 4. Comparison of real radiograph and simulation for a hip joint. 

4. Realisation 

The C-arm has five serially arranged joints, which limit the ability for free positioning in 3D 
space to 5 DOF. Image-plane and -center can be selected freely but the rotation around the 
beam axis depends on the first parameters. 

The challenges of the hardware part of robotization lie in the movement of big masses 
with high precision and OR acceptable speed. Thus, careful selection and integration of 
motors, controllers and encoders is important for the success and are therefore subject 
of discussion in this study. 



4.1 Proof of concept study 

Basis of our work was a C-arm with motorized lift (joint one) and arm-stretch (joint three). 
Position encoders were included for these joints, too, but also for the orbital movement (joint 
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five). Communication to an integrated PC via SSI allowed for isocentric movements in the 
vertical plane (Koulechov et al. 2005). 

For our project, a fully robotized device was required (Binder et al, 2005). In a first step, we 
measured the forces and torques during acceleration and motion. On this information, 
drives, gears, and encoders were selected. Due to lack of space inside the device, external 
mounting was favoured. This also guarantees a high flexibility in changing and adapting 
single components for improvement throughout the development process. 
Communication to the control PC was established via a fieldbus system and a 
programmable control-unit on a PCI-card. The existing communication had to be replaced 
as it could not fulfill the requirements of our system and did not offer the same comfortable 
interface for programming. 




Fig. 5. Joints two, four and five with external drives, gears and encoders. 

After implementation of some test routines, the applications were transferred from 
simulation onto the real system. The tests were successful and indicated the high potential 
of the motorized system (Binder et al. 2006). Some of the results are shown in the 
application-section. Knowing about the precision required for positioning for some of our 
applications like 3D reconstruction, we developed a setup that allowed for registrating the 
real C-arm to the kinematical model. Positions and orientations were measured by an 
infrared tracking system and then compared to those we calculated. 



4.2 Experiences 

Positioning the C-arm with our software can be done easily and shows that the idea will 
work out. Therefore it is sure that a minimum of costs and effort can help to improve the 
workflow in mobile X-ray imaging. 

Nevertheless, there are still some problems to overcome. The weights which have to be 
moved here require a high gear reduction. Moving the joints manually is therefore hard 
to manage as it requires a lot of strength, that would damage the gears. We will 
integrate a different coupler in the next generation. Meanwhile the semi-manual 
handling, i.e. motor-assisted movements, which were in the first place integrated for 
weight compensation, performs very promising and will be part in the next generation, 
too. Additionally, effects of mechanical deformation and torsion during positioning are 
too big, too, and cannot be ignored. Right now we are working on several methods to 
compensate for these errors. The results will be published later. High masses in 
combination with friction also influence the positioning accuracy. Especially joints two 
and four are subject of further improvement. For better motor-controlling and to 
minimize effects of torsion and elasticity in the drive-axis and belts, the position 
encoders will be positioned at the driven part as originally planed. 

The next generation of our C-arm will integrate the experiences we have obtained and allow 
for further evaluation. 
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5. Kinematics 

Now that the C-arm joints can be moved motor driven, some of the applications such as 
weight compensation can already be realized. But performing exact positioning requires 
knowledge about the relationship between joint parameters and C-arm position and 
orientation. This so-called kinematic problem consists of two parts. The direct 
kinematics, i.e. calculating position of the ROI and orientation of the beam from given 
joint parameters can be derived by setting up the DH-matrices and multiplying those 
one after another. These basics are described e.g. in (Siegert, HJ. & Bocioneck S. 1996). 
For most applications ROI and beam direction are given from the medical application 
and with respect to the C-arm base or a previously taken radiograph. The joint- 
parameters for this new position have to be calculated. This is the so-called inverse 
kinematic problem. In the following section we want to introduce a geometry based 
solution. 

5.1 Inverse kinematics 

Conventional C-arms have five joints and thus are limited to 5 DOFs. The consequence 
is that the rotation of the radiograph around the center beam depends on the position 
p of Z (=ROI) and the beam direction m z which defines the z-axis of our tool 
coordinate system. From the transformation matrix M we therefore know only those 
two vectors: 
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Table 2: Information given for the inverse kinematics, introduces the basic variables in this 
calculation. They are illustrated in Fig. 2 and Fig. 7. Axes Fig. 7 Axes and vectors used for inverse 
kinematics. 
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Fig. 6. mechanical offsets of the system. o 4 is Fig. 7. Axes and vectors used for inverse 
the rotational center of the C and 5 the kinematics, 
center between image intensifier and 
generator. 
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Ko 


Base coordinate system with x yoZo 


Zo 


Translational axis of joint 1 rotational axis of joint 3 


5 


Midpoint between XR and II. 


Z 


Target (= ROI), o 5 = Z in case the default radius is used. o z can be moved 
along the center beam to adapt the radius XR <-> R0 1 . 


m z 


Beam Direction 


g z =O z + s-m z 


Center beam of x-ray cone with direction m z through ROI Z 


m 3 


Direction of arm, parallel to XqYo -plane Rotational axis for joint 4 


g 3 = h3 + t-m 3 


Line along arm at height 61 with direction m 3, intersecting g z 



Table 2. Information given for the inverse kinematics. 



The idea of this calculation is now to reduce the number of unknown joint parameters by 

applying geometrical knowledge and inter dependencies. 

As illustrated in Fig. 7., the translational axis (J3 with direction vector ffi3 describing the 

arm, depends directly on @ 2 . Equating g 3 and g z allows for expressing the height of the 
arm d x as a function of @ 2 : 

Q,=0 2+ t-m 3 

-sin(0 2 n \ A ) 

+ t- 







COS(0 2 ) 





9z =0 Z +s-m 2 
vOzzy 



m Z) 
m Z) 

v m z Z ; 



di 



m zx +m zy tan(02) 





(3) 



(4) 



Fig. 8. 5 is derived in the C-plane on Fig. 9. @ in the mechanical system, looking 
base of g and g z . along g towards the C-arm base. 
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These functions can now be used to calculate the joint parameters for A4 and A5. Two 
planes have to be defined for further work: The C-plane e with normal vector fl c and the 
7 n -plane with normal-vector n : 

L 0\i3 r M 03 

nc =h x Qz ( 5 ) 

n 03 = g 3 xzo (6) 

|@ I is the angle between the C-plane and the z g 3 -plane (see Fig. 9). Its sign is decided in 
dependency of n 03 and the part g of g z , which is parallel to n 03 : 



sign(0 4 



Ifor g Z ||nl|n 3 
-lfor g Z ||nSn 3 



(7) 



I© I is given by the angle between (J z and g 3 . To fit into our definitions, this value is 

substracted from -y (see Fig. 8). The sign is fixed by extracting the part g z p from g z , 
which is parallel to g 3 : 



sign(© 5 ) = 



lfor g z p II 93 
-lfor g Z || 3 #g 3 



(8) 



The last missing joint parameter is the length of the arm, i.e. the horizontal distance between 
O2 and O3 (see Fig. 10). Due to the mechanics, the following information is given: 



[05,04] 


Perpendicular to g z , length as, in E c , 


[o 4 ,o 3 ] 


Perpendicular to g 3 , length a 4 , in E c , 




Fig. 10. Projection into the E q plane. 

To get the correct directions vectors, the vectors are defined: 



maiz 


Part of 1% perpendicular to m z 


m z ±3 


Part of m z perpendicular to ITI3 
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Now starting at O5, O4 and O3 can be calculated: 

04 = 05-85-1^ (9) 

03 = 4 -a 4 -m zl 3 (10) 

The armlength is then: 

d 3 = A /0^V ( n ) 

As defined at beginning of this section, 3 gv , z gv , and all their parts are functions of 2. 
Therefore the calculated functions for the joint parameters di, d 3 , 4 , and 5 depend on 2, too. 
When inserting them into the homogeneous transformation matrix %{6 b & 2 .6 3 .e A .e 5 ) » we 9 et by 
symbolic calculation of the direct kinematics, we get o A5 ( 02 ) . 

The correct 2 is found by determining the zero of the difference function between the 
position p(@ 2 ) and orientation z ( @2 ) from °a 5 (© 2 ) an< ^ * ne gi yen target parameters p and Z . 
This can be done by numerical means. 

There is one case which has to be dealt with separately. For 5 = -gCF , ffi z is anti-parallel to 
ITI3. The lines have the distance d 4 5 = a4 + a 5 and are parallel to the XqYo -plane, i.e. m zz = 0. 
As they do not intersect, the approach introduced above cannot be used. When starting the 
inverse kinematics an intersection cannot be determined and © is still unknown. Checking 

for m zz = is a necessary constraint but also includes all 5 for © 4 = ± n . This can be filtered 
out by checking the distance d between z and g z . As illustrated in Fig. 11, the maximum 
is given by d . For d >d 45 the calculation can be done as mentioned above, otherwise, the 
following solution has to be used. 

Projection into the x yo -plane simplifies the calculation, d is the distance between fn and 
rfi when projected into this plane. 4 can now be derived using d and d 45 . The sign of 4 
is determined by the intersection I of m z with the x -axis. 

04=^r[Al (12) 

sign(© 4 )J+; f !*« (13) 

[-1 for i>0 

d 3 is calculated from d and I using the theorem of Pythagoras. Now the value of @ 2 can 
be derived. 
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2 =£-sign(0 4 ) (16) 
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Fig. 11. Special case: © 5 = -QO 9 • 
6. Applications 

When this project started, the major goal was to simplify the handling of the C-arm. The 
user should be able to reach the intended target position in terms of a point and shoot 
procedure known from photo cameras. Thinking about which joint has to be moved in 
which way to reach the right position should no longer be necessary. Positioning has to be 
performed by means of the radiographs taken and in terms of left/ right, up/ down, 
forward/ backward. This way we wanted to get away from the trial and error solution. 
Nevertheless, the common way of handling the C-arm has to be kept. Several reasons have 
to be mentioned for that; Safety in critical situations, "backward-compatibility" for people 
who want to get used to the system step by step and finally for those users, which don't 
want to use a robotic system for whatever reason. 

In the following sections, we give a number of examples that can already be performed on 
the current version of our robotized C-arm. We achieved not only simplified positioning, 
but also opened a way for completely new applications. Due to the close cooperation with 
surgeons, more will follow, soon. 

These examples also make clear, that the staff in the operation theatre will be in full control 
of the device at every time. None of these applications has to be performed in an assisted 
way, as the common means of handling are still available and override all other actions. 
However, if the user wants to use one of the additional functions offered, they are available. 

6.1 User Controlled Movements 

6.1.1 Semi-manual positioning 

The moving parts of the C-arm are weight-balanced and thus can be moved easily. Nevertheless, 
in some positions the manual acceleration of image-intensifier and generator requires some 
strength. Applying the drives, gears, and couplers for the robotized system increases friction. To 
ease the effort, the user-intention for the movements of each single joint is recognized and the 
drives move accordingly. Thus, a simple weight compensation is realized. 

As already mentioned above, purely manual positioning is not possible at the moment, but will be 
available after some changes in the mechanical system in the next generation. Even in the worst case 
scenario of a power failure, the C-arm can be moved away from the patient to ensure a safe surgery. 

6.1.2 Cartesian Movements 

When operating the standard mechanical C-arm by hand, joint parameters were changed 
one after the other. The non-trivial architecture of the fluoroscope leads to complex changes 
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of the region of interest and the viewing direction when moving a joint. Thus it is difficult to 
set the parameters of the C-arm for a certain region of interest. Using the inverse kinematics, 
movements along Cartesian axes can be performed easily. Those motions are much easier to 
visualize by humans and allow for faster positioning. Here it makes no difference which 
coordinate system is used. The translations and rotations can be specified in relation to the 
C-arm, to the patient, or to the radiograph. The latter allows for shifts in the image plane, in 
order to recenter (coordinates x and y) or to obtain a closer view (coordinate z). The new 
target position can be marked directly on the radiograph, e.g. on a touch-screen. To make 
positioning even easier, a simulation of the new image - based on previous pictures - can be 
presented to the OR staff on the computer screen. 

6.1.3 Isocentric Rotation 

As already noted, common C-arm designs were not optimized for isocentric movements due 
to manufacturing reasons, i. e. when moving joints four and/ or five, the image center will 
change. This complicates image acquisition of a ROI from different viewing angles. 
Unlike other systems Koulechov K., 2005, Moret, J, 1998), we are not limited to one fixed 
plane or a fixed distance to the center of rotation but can reach all possible positions in 
between the mechanical limitations. Using the inverse kinematics, we can easily rotate the 
C-arm with a user specified distance around an arbitrary point and in an arbitrary plane. All 
the surgeon has to do is to define the center of the rotation, either by marking a point in an 
image - an fast, but somewhat inaccurate way - or by marking the same landmark in two 
different images to allow for the extraction of 3D-coordinates of the desired fixed point - the 
exact way. The plane of the rotation can be defined by the plane of the C of the fluoroscope 
or by defining any two linearly independent vectors centered on a fixed point. 

6.2 Automatic Positioning 

6.2. 1 Re-centering 

Placing the C-arm in such a way that a given ROI will be imaged is often difficult and 
requires experience, especially when the patient is corpulent. If for example an image of the 
femur head has to be obtained, its center should be in the center of the radiograph. If an 
image is not centered properly and the ROI is on the edge of the radiograph, important 
information might be cut off. 



_ -w 

n 




Fig. 12. Selecting the new center on an misplaced radiograph for automatic repositioning. 
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Nevertheless, the picture can still be used to obtain a corrected image. After marking the 
desired center in the old image, the robotic C-arm can automatically reposition itself such 
that the desired region is mapped at the center. The viewing angle can either be corrected 
before taking the new image or be kept from the old image. 

6.2.2 Standard Images 





Fig. 13. Defining landmarks on two radiographs for exact automatic a.-p. positioning on a 
vertebra. 

Many radiographs are views of standard planes e. g. anterior-posterior, lateral, medial etc. A 
motorized C-arm can be moved easily to these positions. In order to enable the robotic C- 
arm to move to the desired position, the surgeon identifies several landmarks on two 
pictures of the ROI taken from different angles. The 3D-coordinates of the landmarks are 
calculated internally and the correct image center and image direction are calculated based 
on the inverse kinematics. The latter one relies on marking special landmarks from which to 
get the right anatomical position for the image. E. g. for a-p-images of the spine the surgeon 
would mark the left and right transverse process and the spine of the vertebra to define the 
region of interest as well as the a-p-direction (see Fig. 13). It should be mentioned that other 
image modalities can be easily included into the set of standard images by defining the 
number of points to be marked and the algorithm describing the ROI and beam direction in 
terms of the marked points. 

6.3 Image Sequences 

One of the most interesting perspectives for a motorized C-arm are applications which need 
the input of more than one image. These applications require an exact calibration of the 
generated image (Brack et al. 1996 & 1999)* and compensation of the mechanical 
deformations caused by the masses of the source and the detector. Due to the integrated 
position encoders for each joint, position and orientation of each image is known. Therefore, 
we also know the relationship between all images taken (as long as the C-arm base is not 
moved). 



6.3. 1 Longbone And Poster Images 

Some structures in the human body are bigger than the field of view of the fluoroscope. In 
orthopaedics, sometimes images of the whole femur are useful. Neurosurgeons may have to 
scan the whole spine to find the vertebra of interest. Images of the entire thorax or pelvis are 
important in emergency rooms or surgery. Currently the field of view of the standard C-arm 
is very limited (a 17 cm diameter circular image is typical). Our solution is to acquire several 
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images and compose a single image by (semi-) automatic positioning of the C-am during 
image collection. By marking the starting point on two images from different angles its 
position in space can be calculated. After defining a target point in the same way, the angle 
of view (a-p, lateral or user-defined) is set. Depending on the distance between start- and 
end-point, the C-arm will take a series of radiographs and will align them to one single 
longbone-image. Our longbone-application offers user-interactive image acquisition. 




Fig. 14. Radiographs and combined longbone-panorama from real experiments with the C-arm. 



It is a non-trivial task to align single images for manual long-bone-image construction. 
Different methods, such as X-ray rulers (Yaniv Z. & Joskowicz L., 2004) were suggested for 
registration. In our approach, position and orientation of the images in space are given by 
the position encoders and are saved with the image data. Therefore, it is not difficult to 
combine the images with a minimum effort of registration. The result shown in Fig. 14 was 
reconstructed with the given joint parameters only, i.e. no additional registration was 
applied. 

For emergency radiographs, marking start- and endpoint in two images each will take 
too long. Thus, only the start point and a direction-vector are marked in the first image 
and the fluoroscope moves in the indicated direction parallel to the image plane taking 
a specified number of radiographs. Additionally, the direction of movement might be 
corrected during processing. 

An application similar to the longbone sequence is taking an image of the whole thorax. 
There, not only one row of images but a whole matrix of images must be taken and 
combined in the correct way. It is easy to see that our approach extends to this application 
naturally. It has to be mentioned that due to the form of the x-ray cone, deep objects will 
achieve projection errors when overlapping the single radiographs. This is a problem which 
all cone-beam machines have to overcome. For single bones such as the femur shown in Fig. 
14, these errors are less than 1 mm. 



6.3.2 Intra-surgical data collection for 3D reconstruction 

In some surgeries, 3D-images of bones or other body structures substantially help to get 
better orientation and control. Acquiring such image data in a standard OR is complicated 
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as a CT is normally not available. With the help of the fluoroscope robot, CT-like 3D-images 
can now be acquired. For this, the C-arm is rotated around the ROI in small discrete steps in 
an isocentric movement. In each step, a radiograph is taken. The rotation plane can be 
selected freely within the mechanical limits of the C-arm. 




Fig. 15. Collecting radiographs for 3D reconstruction. 



A 3D model of the ROI can be calculated with an iterative reconstruction algorithm, e.g. the 
algebraic reconstruction technique (ART) algorithm (Toft P. 1996), in reasonable time. The 
achieved 3D resolution is sufficient for many diagnostic means and rises with the increasing 
resolution of the image intensifier. Unlike (Ritter et al. 2003), about 20-50 images are 
necessary to create a sufficient 3D image of the target. Furthermore, the distance between 
the ROI and the detector-screen can be varied freely and is not fixed. 




Fig. 16. Series of projections and the reconstructed result. 



7. Results and Future Work 

Experiments with this world's first fully robotized C-arm show very promising results. 
Movements in the image plane, longbone images, data acquisition for 3D reconstruction, 
and landmark based positioning were successfully implemented and tested. The next 
version of our C-arm will overcome mechanical problems such as lacking positioning 
accuracy and mechanical deformation. 

The value of an e very-day OR-device can be increased by simple means. The workflow can 
be optimized and mistakes can be prevented. Operation time is saved and radiation dose for 
patient and staff is minimized. The experiences of this study have already been transferred 
to other systems such as OR microscope (Knopp et al. 2004) and showed similar promising 
results. Chances are that the number of such robot- assisted systems in the OR will increase 
soon. 
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1. Introduction 

Facial caricature as a facial image media acts as an interface between the user and the 
computer. We developed the facial caricaturing robot "COOPER" (Fig. 1) that was exhibited 
at the Prototype Robot Exhibition of EXPO 2005 [1]. COOPER watches the face of a person 
seated at the chair, obtains facial images, and analyzes the images to extract 251 feature 
points to generate his facial line drawings with deformation, and gives a caricature drawn 
by the laser pen on the shrimp rice cracker to the visitor. 

We have been developing a facial caricaturing system PICASSO [2], and we customized this 
system for the exhibition at the EXPO pavilion. This paper illustrates the outline of the COOPER 
and the details of the image processing fabricated in it. And we discusses on the COOPER' s 
prospects of the future subjects based on more than 395 facial caricatures obtained at EXPO2005. 




Fig. 1. External view of COOPER. 



Though there are other systems and products proposed so far that can generate the caricature 
automatically [3-5], there has been no report on the field test of these systems. Therefore our large 
trial can also be expected to get facial images covering several races and generations in the grand 
field test at EXPO. Therefore it would be unique and effective to illustrate what COOPER is and 
how it performed at EXPO for prospecting the future of computer facial caricaturing. 
As a result of 11 days demonstration, COOPER generated 352 caricatures in total and presented a 
shrimp rice cracker to the respective visitor. This system generated successful output from the 
standpoint of wide distribution of visitors in generation, race and sexuality at EXPO site. 
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In section 2, the outline of COOPER system is summarized, and in section 3, the image 
processing methods for extracting facial features are introduced. In section 4, the primal 
experimental results of the exhibition are presented. Some knowledge and future subjects 
obtained by the considerations of results are shown in sections 5 and 6. 



2. Configuration of COOPER 

2.1 Line drawings for facial expression in COOPER 

COOPER draws the facial caricature on the shrimp rice cracker (Fig. 2) with laser pen 
held in the left hand. This facial data F (defined by Eq. (1)) has 251 feature points and 
this system expresses the facial caricature with the 2D line drawing by connecting these 
feature points. COPPER and the original facial caricaturing system PICASSO extract 
some facial individuality features from the input face and deform these features to 
generate its caricature as follows: The facial caricature Q is generated by comparing the 
input face P with the mean face S, which is defined previously by averaging input faces 
as shown in Fig. 2 and Eq. (2). This system generates the facial caricature with some 
exaggeration by specifying its rate b for adjusting the deformation of the caricature to 
each visitor. 



F = {(x,.,y,.)|/ = 1,2,...,251] 
Q = P + b(P-S) 



(1) 
(2) 




Fig. 2. Caricature on the Shrimp rice cracker. 





(a) Mean face 
Fig. 3. Example of caricature generation. 



(b) Caricature 



Facial Caricaturing Robot Cooper with Laser Pen and Shrimp Rice Cracker 
in Hands Exhibited at Expo2005 



421 



2.2 Hardware modules 

An exterior view of COOPER is shown in Fig. 1. For capturing the facial image, a CCD 
camera is mounted in the right eye of the head, and a pair of industrial-use robot arm 
modules are fabricated as the right and left arms. The laser pen mounted at the left arm was 
connected to the body by the tube type glass fiber not to be fragile due to the arm motion. 
Laser power was controlled so as to adjust the distance from the pen to the shrimp cracker 
and surface conditions (temperature, moisture, materials and roughness, etc.) of the cracker. 
The velocity and the power of the laser pen were controlled smoothly in order to draw the 
caricature even if the surface of the cracker is rough and distorted. 

After many preliminary experiments, we decided the following such conditions that the 
moisture of the cracker is around 11 % and the surface color of the cracker is brown or black. 
For the security of the visitors from the laser pen hazard, we fabricated a metal box which 
covers the laser pen, as shown at the bottom right of Fig. 1. 

The industrial-use robot arms were assembled together with the head with cameras, body 
and legs. Since COOPER has both tilting and rotating mechanisms of the head, we could 
design the motion of the robot head to be performable like a human caricaturist. We also 
designed the motion of arms to reduce its loading weight as small as possible and then to 
realize smoother movement of the arms. We took also the safety conditions into 
consideration in order to cope with some abnormal operations of the laser pen. System 
configuration of the robot, control panel and image processing PC is shown in Fig. 4. 
We took some auxiliary information of the respective visitor by using touch-sensitive panel 
as shown in Fig. 5. The contents of this information consist of a kind of facial expression 
(normal, sad and smiling), sex, age(less than 10, 10's, 20's, 30's, 40's, 50's, more than 60), and 
his authorization for the usage of his face data for further researches on facial caricaturing. 



Robot control panel 



Image processing PC 



"- CCD camera r 



Right arm module 



1 



Left arm module 



Fig. 4. System Configuration. 




Fig. 5. Touch-sensitive panel and blue backrest chair. 
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2.3 Software modules 

For obtaining the images, the visitors are asked to sit at a chair with the blue backrest, as 
shown in Fig. 5. This system extracts the facial features from input images, and generates the 
caricature in the same way as the deformation method of PICASSO system. This system 
uses a couple of facial images captured by CCD camera with 1 fps, because visitors often 
close the eyes. If this system fails to process the first image, the same procedure is applied 
again to the second image. Then, COOPER generates the caricature, converts it into the 
robot control language and finally controls the laser pen. 

In this facial image processing, this system first extracts irises and nostrils, and 
afterward defines the regions of eyes, nose, mouth and ears hierarchically [6]. This 
system extracts also the hair region and skin color region in addition to the facial parts 
features. In the final result, this system generates the caricature comprising 251 feature 
points that are defined originally as the cooper-PICASSO format. This system 
evaluates simultaneously the quality of the intermediate results including caricature 
by using "fail-safe modules' 7 as shown precisely in section 3.5. Thus, we have 
succeeded in designing the robust performance even in the unconditioned 
circumstance of illumination in EXPO site. 



3. Detailed procedures of facial image processing 

As the preprocessing for extraction of facial features, this system detects skin color region 
from RGB image. In the preprocessing, blue region of the background is eliminated from 
the input image. Skin color region is eliminated from input image as shown in Fig. 6 based 
on the hue discrimination as shown in Fig. 7. This skin color region is defined and used for 
the successive image processing. 





Fig. 6 Input face. 



Fig. 7 Skin color region. 



3.1 Irises recognition by Hough transform 

In this system, irises are first extracted by using Hough transform [6] for leading other 
hierarchical processing modules. Secondly nostrils are extracted in the same way of irises at 
the nose region. The results of irises and nostrils are shown in Fig. 7. 



3.2 ROI extractions for Facial parts 

The regions of interest (ROI) of eyes, nose, mouth and ears are defined on the basis of the 
information on irises and nostrils. Fig. 8 shows an example of ROI extraction. 
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Fig. 8. Example of facial features extraction. 



3.3 Contour extractions of facial parts 

As defined in each facial parts region, outlines of eyes, nose, mouth and ears are detected at 
the respective ROI of the gray image by using smoothing, contrast improvement, 
thresholding and thinning procedures. We basically designed that the caricature of 
COOPER is represented with a set of line drawings. This means that the face of line 
drawings is less informative than the original image in physical meaning, but that the face of 
line drawings is more effective than the face image in impression. In this sense, the shape 
feature of the face contour, hair and jaw is more dominant than the gray image. Moreover 
the fact that the face of line drawings is easier to realize the correspondence among faces 
than the face images is one of the technical advantages. 

The outline of hair is detected from the binary image by the method of smoothing, contrast 
improvement and thresholding, as shown in Fig. 9. 

The outline of jaw is detected from R image of RGB color image by using Sobel operator and 
thresholding, as shown in Fig. 10. 





Fig. 9. Hair region. 



Fig. 10. Pre-processing of jaw extraction. 



3.4 Details in fail-safe module 

At the same time of the extraction of facial parts, this system evaluates how feasible the 
result is, and modifies the result, if necessary, according to the statistical standard for the 
positional relationship among facial parts. This fail-safe system evaluates the result by the 
estimation function preliminarily prepared [7] which was defined by the difference between 
the result of the input face and mean face. If this system rejects the result, it is replaced by 
the corresponding facial parts of the mean face and fitted it as the facial parts. 
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4. Demonstrations in EXPO2005 

4.1 352 facial data 

We demonstrated successfully that COOPER system could be exhibited at the Prototype 

Robot Exhibition of EXPO 2005, Aichi Japan. COOPER watches the face of a person and 

generates his facial line drawings with deformation. The details of Prototype Robot 

Exhibition are as follows: 

Name: Prototype robot exhibition, The 2005 World Exposition, Aichi, Japan 

Location: The Morizo and Kiccoro Exhibition Center of Nagakute Area 

Duration: Jun.9 to Jun.19, 2005 (11 days in total) 

Number of visitors: 

to EXPO: 1,129,390 

to Prototype robot exhibition: 123,000 
The number of caricatures presented: 352 



4.2 Successful cases with famous visitors 

Many famous people visited our site as television coverage. Fig. 11 shows Japanese famous 
people. 





(a) Maoko Kotani. (b) Mana Mikura. (c) Kana Mikura. 

Fig. 11. Caricatures of Japanese famous people. 



(d) Kumiko Endo. 



4.3 Un-successful cases 

Investigating the experimental results, it was known that a few un-successful caricatures 
were generated due to several causes of image processing processes. The detailed statistics 
of the results will be introduced in subsection 5.1. The examples of the causes of the un- 
successful cases are shown as follows: 

(1) Irises recognition 

Un-successful caricatures were strongly affected by the failure in irises recognition 
due to the un-expected blinking. We, therefore, have to develop such new module 
that could detect the blinking automatically. 

(2) ROI extraction 

Facial parts of ROI couldn't be occasionally extracted at the proper location. We 
must try to improve the method for extracting ROI involving the major part of the 
respective facial parts at least. 

(3) Contour extraction of facial parts 

The contour of facial parts couldn't be frequently detected at the proper position 
due to the existence of noise. We must try to improve the method for robust 
detection of the facial parts contour. 
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5. Considerations and coming subjects 

5.1 Statistics of success rate for facial parts recognition 

We evaluated statistically the experimental results based on the subjective visual investigations 
of more than 10 testees visual judgments. The case of examples of unsuccessful caricature is less 
than 10 percent. Even if un-successful caricature was generated, this system was able to modify 
the caricature to be acceptable to the image processing by using fail-safe module. There is a small 
difference between the successful caricature and the extraction rate of facial feature points as 
shown in Table 1. The row B of Table 1 is the number of successes. The row C and D of Table 1 
are the number of un-successful extraction of facial parts and successful extraction at the 2nd 
frame. Our system worked with stable performance because this system first detects irises and 
nostrils and afterward extracts other facial parts hierarchically. The total number of these 
examples is 77. Thus we succeeded in designing this system to be absolutely fail-safe. Finally the 
number of unsuccessful caricatures became 22, and our system provided the smoother operation 
throughout the whole exhibition. These un-successful cases were caused by the irregular 
direction of face and irregular condition of eyes under the illumination. 
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Table 1 Statistics of caricatures 

(A: Number of generated caricatures, B: Number of successful generation of caricature, C: 
Number of unsuccessful extraction of irises and nostrils and successful extraction of 2nd 
frame, D: Number of unsuccessful extraction of other facial parts and successful extraction 
of 2nd frame, E: Number of generation of fail-safe system, F: Number of unsuccessful 
generation of operators' check, G: Number of fatal failure) 

5.2 Coming subjects and prospects of COOPER 

Although COOPER demonstration in EXPO was almost absolutely successful, the following 
coming subjects were extracted based on the detailed investigations of the experimental results. 
(1) Head tracking 

As shown in Fig. 12, COOPER is likely to fail to capture the adequate facial images 
since the visitor such as a boy often moves in front of the camera. In order to cope 
with this unexpected situation, COOPER must be enforced to be able to track the 
head automatically. 

■I 




Fig. 12. Example of a series of images of moving child in front of camera. 
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To prospect the future of this subject, let us show some preliminary experiments of head 

tracking for acquiring the facial images with sufficient size for the facial parts recognition. 

The outline of the system is shown as follows: 

Fig. 13 shows the configuration of the system composed by a pair of PTZ and fixed CCD 

cameras connected to the host PC system. The fixed CCD camera is to track the head of a 

person who is performing freely in front of it, and the position and the size of the extracted 

head are transmitted to PTZ camera to control the Pan, Tilt and Zoom parameters. CCD 

camera can watch and catch a person's face and PTZ camera captures his facial image with 

sufficient resolution for the image processing in a video rate. 

Consequently, an area where a person can freely perform will be enlarged as shown at the 

dotted region in Fig. 13. 




F'TZ-C.irM-i 
iMWtOOOj 






i ! 

i " * 


PTZ Cant* j 


ton : AW 
Via t+tf. 




£ 


D 


1 


CCD tnmfi.i 


i 


1 


1 



1 ! Covered ArtMi 



ftOflrtun 700mm 
Fig. 13. CCD and PTZ cameras for head tracking. 

By transmitting the location of the head together with the size L of the extracted face region 
to PTZ camera, the vertical, horizontal and depth movements of the head are estimated 
simultaneously and PTZ camera is controlled to stabilize the proper size of the facial image. 
The zoom parameter Z is determined inversely proportional to L by eq.(3). 

Zmax 



Z = - 



-x(Lmax - L) 



(3) 



Lmax - Lmin 

Here, Lmax and Lmin are the maximum and minimum values of this PTZ camera zoom 

specification, respectively. 

(2) Eye contact in facial caricature 
As the facial caricature is a new kind of facial interface media in communication, it is essential 
even in the caricature generation whether the eye gaze expression could be provided or not. 

In order to solve this subject, we have been conducted a preliminary research for generating 

eye-contacted facial image based on the iris recognition. Fig. 14 shows a set of examples. 




Fig. 14. Eye-contacted facial images. 
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(3) Motion in caricature 

Facial caricature could be extended in motion such as smiling, because we have the 

individual facial feature of the motion in emotional expression. 
In order to take this subject into consideration, we have been studied a preliminary 
discussion on the motion exaggeration. Fig. 15 shows an example of motion caricature. 




Fig. 15. Facial expression in motion caricature. 

6. Conclusion 

This paper describes the outline of development of caricaturing robot and the knowledge 
acquired at the presentation in EXPO 2005. It was known that our system COOPER could 
perform successfully at EXPO as the grand field test site. As a result, it was fruitful and 
noteworthy for us that we could collect a large number of facial images from younger and 
middle ages of both male and female. 

However, we should investigate more the intensive evaluation of the caricatures. We try to 
improve the method for the analytical verification of the detailed shape features of facial 
parts. And COOPER is likely to suffer sometimes from the fatal degradations in the feature 
extraction caused by an irregular condition of the facial direction and illumination. We must 
develop robust method for decrease these problems. 

As the future works, we are going to improve these subjects and to exhibit COOPER at other 
events. 
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1. Introduction 

Dolphins are mammalians that integrate the cetacean family. They can be generally found in 
the sea but there are some species that live in rivers. There are some 37 species of dolphins 
mapped in Brazil. Their life time is about 35 years and the period of gestation is 10 months. 
They can reach up to 3 meters in length. The most known, of long nose, are about 2 meters 
long. Dolphins are most known by their abilities of diving and jumping at the water level, 
wondering people. An important characteristic used for distinguishing different individuals 
is its dorsal fin. It has no bounds in its composition and its size varies from specie to specie. 
Most important, its shape varies for individuals of the same specie. Besides this shape 
peculiarity, the dorsal fin keeps marks made by natural attackers, as sharks and others, or 
even fights, that may also be used for distinguishing them. These features form a natural 
signature as a finger tip. 

Currently, there are several researches dealing with these animals in partially demarked 
environments in beaches or rivers. The objectives are such as behavioural studies between 
groups (Link, 2000), counting statistics, and others. One of the difficulties is the lack of an 
exact limit of the observed areas. Besides, there is also some difficulty due to human 
observing capability, which is somewhat avoided due to a constant presence of a same 
specie in a same region. Other known difficulty is the acquisition of dolphin images. Most 
species keeps some distance from boats and other aquatic vehicles or even they disappear to 
deep water. This makes hard the work of researchers who needs to observe a live data or 
data obtained from photos for posterior identification. The last is known as visual photo- 
identification and is often practiced by biologists. Photo-identification can give information 
about motion parameters and population dynamic (Mann, 2000). This method has being 
applied through the last three decades mainly in the Tursiops truncatus (bottlenose) dolphin 
specie. This method is also employed to identification of Orcinus Orca (most known as the 
killer wale) and other cetacean species (Defran, 1990; Flores and Mann, 2000). As the 
pictures are generally taken in different conditions (day, time, and weather), photo- 
identification through purely human observation may be subjected to errors and to human 
analyzer subjectivity. 

The population of Dolphins analyzed in the current work lives in a coastal region of Rio 
Grande do Norte state known as Dolphin Bay, to the south of Natal (state Capital), 
between the Tabatinga and Buzios beaches. The work proposal is to build a system able to 
extract efficient features from digital pictures and to use them for identification of 
dolphins. This is a challenge mainly because pictures are generally taken by biologists and 
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students from distances varying from at least 15 to 40 meters from the animals, being 
some 30 meters the average. Also, the pictures studied are characterized by the presence 
of sea water in almost its entire area but the dolphin, which is generally centralized in the 
photo. 

Efficiency refers to the use of several features in a unique, integrated system able to identify 
the several animals of a family. We remark that there must exist some control mainly in the 
exactly moment in which the pictures are taken, so a somewhat professional, or trained, 
photographer is needed. This is in order to take pictures with the dorsal fin completely 
visible as possible. Photos of joint animals and with boats in it can be treated, but must also 
be avoided as possible, as well as photos with the dorsal fin partially visible. 
We apply two methodologies for identification. The first executes a search in a priority 
space, attributing scores to statistical measures of trustworthiness. The second method uses 
an associative memory based on a multi-layer perceptron trained with an algorithm of the 
type backpropagation (BPNN) (Rummelhart, 1983; Werbos, 1988; Riedmiller and Braun, 
1993). As a new contribution, we added a self-growing mechanism that makes easy the 
inclusion of new individuals eventually discovered in the environment. In this way, feature 
extraction and abstraction combined with the classification algorithm with self-growing 
capability are the new ingredients. These are the system key that makes the system 
robustness, working even in not ideal conditions. For example, as one of the difficulties, the 
worked images are generally obtained with dolphins in different positioning, in different 
angles and in different scales. We developed a normalization approach that partially avoids 
these problems. 

We show experiments and results with images from a database of the Biology Department 
of Federal University of Rio Grande do Norte. The main objective is to define or to learn 
characteristics for some specific animals from this database and to later recover positive 
identifications for these animals from digital pictures of them, making easier the biologist 
work. We remark again that the system deals with already known and also with new 
individuals eventually found in the working region. 

2. State of art 

We found a reduced number of works in the literature about analysis of cetacean fins. 
Between them, we cite the works of Aarabi et al (2000a; 2000b; 2001) in which, respectively, 
string matching and invariant models for photo-identification of dolphins are evaluated. 
From the last work, it is concluded that methods based on curvature analysis of the dorsal 
fin are much more efficient. Differential, semi-differential, and algebra invariants are used. 
A problem found is that digitalization errors may substantially compromise the results in 
the case of using an algebra differential model. The use of semi-differential in variance is 
practically viable, since the number of derivatives as well the number of points used for 
correspondence be limited in two. Hillman (1998) and Kreho (1999), working together, 
experiment two possibilities. In the first, the fin curvature is reduced to a set of Fourier 
coefficients and some similarity invariants are extracted producing as a result invariance 
measures in relation to camera position, angle, and dolphin distance. The other technique is 
based on the search for holes using a rigid curve (a stiff snake), through a four degree 
polynomial that determines an approximation for the fin curvature. Following, the 
difference from this curve to the real points of the dorsal fin is computed. The last technique 
has produced a more promissory set of features. 
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This idea is reinforced by Cesar and Costa (1997) that explain how energy curvature 
diagrams of multi-scales can be easily obtained from curve graphics (Cesar, 1996) and 
used as a robust feature for morphometric characterization of neural cells. That is, the 
curvature energy is an interesting global feature for shape characterization, expressing 
the quantity of energy necessary for transforming a specific shape inside its most down 
energy state (a circle). The curvegram, which can be precisely obtained by using image 
processing techniques, more specifically through the Fourier transform and its inverse, 
provides a multi-scale representation of digital contour curvatures. The same work also 
discusses that by the normalization of the curvature energy with respect to the standard 
circle of unitary perimeter this characteristic gets efficient for expressing complex shapes 
in such a way it is invariant to rotation, translation, and scale. Besides, it is robust to 
noise and other artefacts that imply in image acquisition. 

Just recently, Gope et al (2005) have introduced an affine curve matching method that uses 
the area of mismatch between a query and a database curve. This area is obtained by 
optimally aligning the curves based on the minimum affine distance involving their 
distinguishing points. 

From all observed works, we could see that the major difficulties encountered by researchers is 
in the comparison between several individuals photographed in different situations, including 
time, clime, and even different photographers. These and other issues insert certain subjectivity 
in the analysis done from picture to picture for a correct classification of individuals. 
Photo analysis is based mainly in the patterns presented by the dorsal fin of the animals. 
That is, identification has been carried out mostly by identifying remarkable features on the 
dorsal fin, and visually comparing it with other pictures, following a union of affine 
features. Effort and time spent in this process are directly proportional to the number of 
pictures the researchers have collected. Yet, visual analysis of pictures is carried out by 
marine researchers through use of rules and other relative measurement forms, by hand. 
This may classify individuals with little differences as the same or else to generate new 
instances for the same individual. This problem may affect studies on population estimation 
and others, leading to imprecise results. So, a complete system that starts with picture 
acquisition, to final identification, even a little helped by the user, is of capital importance to 
marine mammal researchers. 

3. The proposed system architecture 

Fig. lFig. 1 shows the basic architecture of the proposed system. It is basically divided into the 
shown 7 modules or processing phases. Each of these phases will be briefly described in the next. 
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Fig. 1. Architecture of the proposed system. 
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3.1. Acquiring module 

The acquired image is digitized and stored in the computer memory (from now we refer to 
this as the original image). An image acquired on-line from a digital camera or from a video 
stream can serve as input to the system. As another option, in the main window of the 
system shown in Fig. 2, a dialog box may be activated for opening an image file, in the case 
of previously acquired images. 
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Fig. 2. Main window of the proposed system. 
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Fig. 3. Acquired image visualized in lower resolution in the system main screen. 



3.2. Visualization and delimitation 

In this phase, a human operator is responsible for the adequate presentation of the interest 
region in the original image. This image is presumably to be stored in a high resolution, with 
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at least a 1500 x 1500 pixels size. The image is initially presented in low resolution only for 
making faster the control by the operator, in the re-visualization tool. All processing are done 
at the original image, without loosing resolution and precision. As a result of this phase, a sub- 
image is manually selected with the region of interest in the original image, containing 
dolphins, and mapped in the original image. This sub-image may be blurred to the user, so it 
would be nice to apply a auto-contrast technique for correcting this (Jain, 1989). 

3.3. Preprocessing 

In this module, the techniques for preprocessing the image are effectively applied. Between 
these techniques, the Karhunem-Loeve (KLT) (Jain, 1989) transform and the auto-contrast 
are used. The KLT transform is applied in order to make uncorrelated the processing 
variables, mapping each pixel to a new base. This brings the image to a new color space. The 
auto-contrast technique is applied in order to obtain a good distribution for the gray level or 
RGB components in case of colored images, adjusting thus the image for the coming phases. 

3.4. Auto-segmentation 

A non supervised methodology through a competitive network is applied in order to 
segment the image based on its texture patterns. This generates another image which has 
two labels, one for background and another foreground (objects) respectively. In order to 
label the image, we use the average of the neighbor values of a selected pixel as image 
attributes for entering the competitive net. It is calculated the mean for each of the 
components, R, G and B, thus giving three attributes for each pixel. 

3.5. Regularization 

In this phase, several algorithms are applied to regularize the image provided by the 
previous phase. This regular, standard image is important for the next phase, mainly for 
improving the feature extraction process. A clear case of the real necessity of this 
regularization is the presence of dolphins in different orientations from one picture to 
another. In the extreme case, dolphins may be pointing to opposite senses (one to left and 
other to right of the image). For this regularization, first the three ending points of the dorsal 
fin, essential to the feature extraction process, are manually chosen by the operator. Note the 
difficulty to develop an automated procedure for the fin extraction, including segmentation, 
detection and selection of the ending points. We have made some trials, ending up with the 
hand process, as this is not the main research goal. In order to get the approximate 
alignment of the dolphin, the system presents a synthetic representation of it (that is, a 3D 
graphical dolphin) whose pointing direction can be controlled by using keyboard and 
mouse. This tool is used by the user to indicate the approximate, actual orientation 
(direction and sense) of the dolphin in the original image. In practice, the user indicates, 
interactively, the Euler angles (roll, pitch, yaw) relative to the approximate orientation. 
These angles are the basis for defining the coefficients of a homogeneous transform (a 3D 
rotation) to be applied in the 2D image in order to approximately conform to the desired 
orientation in the model. Then the image is ready for feature extraction. 

3.6. Morphological operators 

Mathematical Morphology techniques tools are used for extracting boundaries, skeletons, 
and to help improving other algorithms. Mathematical morphology is useful for improving 
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extraction of shape features as the fin, besides being the basis for algorithms for curvature 
analysis, peak detection, between others. 

3.7. Feature extraction 

In this module, some features representing the dorsal fin shape are extracted, as: height and 
width, peak location, number and position of holes and/ or boundary cracks, between others. 
These features will be better presented and discussed next, in Section 4 (Feature extraction). 



3.8. Identification 

The extracted features are then presented to the classifier, being given as result an answer 
about the correct identification or not of the dolphin. As a new tool, we added one of the 
methodologies for classifying with a self-growing mechanism, that can even find new 
instances of dolphins, never met previously. 




Fig. 4. Dolphin with substantial rotation around Y axis. 



Original 
Image 



■*- KLT +- Auto contrast ■*- Auto segmentation 



Thresholding ■*• Reflexion -► Cut -► Labeling 



Extract 
Fin 



-► Skeleton ■* Boundary -► Extract Peak 



Sequencing 
Points 



Detecting 
Holes 



Obtain Curves 
Parametric 



-► Normalize 



Fig. 5. Proposed sequence for pre-processing images. 
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4. Feature extraction 

Water reflection makes texture features vary substantially from a picture to another. That is, 
the quantity of intensity is not uniform along the fin, thus even changing local characteristics 
extracted in relation to other positions in the fins. In this way, the fin shape, number and 
positioning of holes/ cracks, are less sensitive to noise and intensity variations. Of course, there 
are also restrictions. Lets consider a left handed 3D system with its origin in the centre of 
projection of the camera (that is, with positive Z axis pointing through the picture). Fig. 4 
shows an animal that represents a relatively large rotation in relation to the Y axis. In this case, 
even if the fin appears in the picture with enough size and has several holes, most of them may 
not be detected. So, curvature features extraction may be strongly affected. In this way, it 
would be necessary to apply a series of processing that preserves and enhance them at mostly. 
The processing sequence proposed to be applied for this is shown in Figure 5. 



4.1. Preprocessing 

As introduced previously, the image is initially processed with the application of KLT 
transformation to enhance desired features. Results of KLT is an image with two levels of 
intensity, we adopt and 50. Using these values, the resulting image is further binarized (to 
or 1 values). The image is then cut (pixels are put to 0) in the region below the line defined 
by the two inferior points of the fin entered by the user. Note that we assume the y axis of 
the camera in vertical alignment, then the interest region of the fin must be to the top of that 
line. So the points below it are considered background. The image is reflected (in 3D) 
around the y axis, that is, only reconfiguring the x coordinates. This is already a first action 
towards regularization for feature extraction. Subsequent process of image labelling is 
fundamental for identification of point agglomeration. Vector quantization is used for 
splitting the image in classes. Due to noise present on it after the initial phases, it is common 
the existence of several objects, the fin is the interest object. A morphological filtering is 
realized to select the biggest one (presumably the fin) to the next processing phase. 





















/^ 






/^ 






\ - 




^.. x 




A 




^J 


--^ 

















Fig. 6. Fin peak extraction from the image. 

At this time, a binary image with the fin (not yet with rotation corrected) is present. Border 
and skeleton images are then extracted using image processing algorithms. These images 
will be used for extraction of the peak using the algorithm graphically depicted in Fig. 6. 
Initially, it extracts two sparse points on the top portion of the skeleton, some few pixels 
apart of each other. These points define a line segment which is prolonged up until it 
intersects the fin boundary. The intersection point is taken as the peak. Besides simple, it 
showed to be a very good approximation to the peak, as the superior shape of the skeleton 
naturally points to the peak. 
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4.2. Sequencing the fin points 

In this work, the features used for identification can be understood as representations of the 
fin shape. So, before feature computation, we first extract and order pixels on the fin 
boundary. Border pixels in a binary image can be easily extracted using morphological 
morphology. The boundary is given by border points following a given sequence, say 
counter-clockwise. So, we have just to join the points finding this sequencing. The algorithm 
starts from the left to the right of the image. Remember during user interaction phase the 
approximated initial (pi), peak (p), and final {pi) points are manually given. Based on the 
orientation given by these points, if necessary, the image is reflected in such a way the head 
of the dolphin points to the right of the image. A search on the border image is then 
performed from pi to pi to find the boundary, that is, in crescent order of x. If a substantial 
change is detected on the y actual value in relation to the previous one, this means the y 
continuity is broken. So the search is inverted, taking next x values for each fixed y. When 
the opposite occurs, the current hole has finished, the algorithm returns the initial ordering 
search. In this way, most boundary points are correctly found and the result is a sequence of 
point coordinates (x and y) stored in an array (a 2D point structure). Results are pretty good, 
as can be seen in Fig. 7. It shows the boundary of a fin and the sequence of points obtained 
and plotted. 




Fig. 7. Extracting fin boundary points from border pixels of the image (borders are in the 
left, boundaries are in the right of Figure). The method may loose some points. 



4.3. Polynomial representation of the fin 

For curvature feature extraction, we propose a method that differs from those proposed 
by Araabi (2000), Hillman (1998), and Gopi (2005), cited previously. Our method 
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generates two third degree polynomials, one for each side of the fin. The junction of these 
two curves plus the bottom line would form the fin complete shape. This method has 
proven to be robust, with third degree curves well approximating the curvature. Further, 
we can use these polynomials to detect possible holes present on the fin in a more precise 
way. Fig. 8 describes the parameterization of the curves. Note that the two curves are 
coincident for X=0. This point is exactly the fin peak. The parametric equations of the 
curves are expressed as: 

x 1 U) = a x / + b Xi l 2 + c x / + d Xi 
y 1 U) = a yi ^ 3 + b yi l 2 + c y / + d yi 

Curve2^ 2 ' * ,* 2 

y 2 U) = a y2 ^ 3 + b y2 ^ 2 + c y2 ^ + d y2 

x 1 (0) = x 2 (0)=>x 1 (0) = A X2 



Restrictions 



x 2 (0) = d x 



d =d v 



In this way, one needs only to solve the above system using the found boundary in order to 
get the parametric coefficients in x and y. A sum of squared differences (SSD) approach is 
used here, with the final matrix equation simply given by AC x = X . An equivalent approach 
is also adopted for the second curve. 

^ X 



1=0 




curve 1 



1=1 



curve 2 



X=l 



Fig. 8. Graphical model of the parametric curves used. 



4.4 Obtaining images with regularized (standard) dimensions 

The camera image can be assumed as a mapping from the world scene in a plane, a 2D 
representation. Also, we must consider that the dolphin may be rotated in the world, we 
use Euler angles (x, y, and z rotations are considered). Besides, we can do some important 
simplifications. The first is not to consider x rotation as in practice we noted its maximum 
value is less than 10 degrees. Lets consider the dolphin body (its length) is distributed 
along the x g axis, with its head points to positive direction. The width of the animal is 
distributed along z g , with its center at the origin z g = 0. The height of the dolphin is along 
the y g axis, with the fin peak pointing to positive values of this axis. The second 
simplification is that, considering the animal at some distance, the y coordinate of a point 
P, say P yr rotated around the y axis and projected at the image plane will be the same as if 
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the transformations does not happen (P y ). It is easy to understand and verify this 
assumption if we analyze the effects on an object close enough and far enough of an 
observer. If the object is close, a rotation 9 around y would affect the y coordinate (the 
closer to n/2, the bigger change in y). But, if the object gets far from the observer, the 
displacement in y decreases. The perspective can be neglected and/ or assumed as a rigid 
body transformation. Ideally at infinity, the y change would be zero. As in this work 
pictures are taken at a distance varying from at least 15 to 40 meters (average 30), the 
change in y is so small in comparison with the dolphin size and can be neglected. Note 
that a rotation and projection is performed here. 
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Fig. 9. Mapping between 3D and 2D 

Given a picture in the XY plane with z = 0, the goal is to get an approximation for the 
coordinates of the real points of the fin on the world, mapped back to 2D. We now can 
consider that the fin is parallel to the observer plane without rotation. Fig. 9 can be used to 
better understand the model. Given the Euler angles (rotation) and assuming the 
simplifications suggested above, equations can be easily derived for determining the 
mapping between a generic image and the one on a regularized configuration. These angles 
are interactively given by the user when he rotates the graphical dolphin shown in a 
window to conform with the configuration seen on the image. 

We remark that these transformations are applied to the parametric equations of the two 
polynomials that represent the fin. In this way, the equations representing the ideal fin 
(without any transformation, aligned to the x axis) are found. After these transformations, it 
is necessary to normalize the equations as varied sized images can be given, as explained 
next. 



4.5. Polynomials normalization 

After all pre and post-processing, a normalisation is carried out on the data. This is of 
fundamental importance for extracting features that are already normalized, making 
analysis easier, as the images of a same animal coming from the data base may be so 
different. As stated above, the normalization is done at the coefficients that describe the two 
parametric, cubic curves. This normalization process is depicted in Fig. 10. 
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Fig. 10. Sketch of the algorithm for normalization of the polynomials. 



4.5.1 Shape features (descriptors) 

After polynomial normalization, shape features are then extracted. One of the most 
important features is the fin format, so a notion of // format ,/ must be given. Two 
features are extracted for capturing the format notion: the curvature radius of the two 
curves and the indexes of discrepancy to a circumference. These indexes measure the 
similarity between one of the sides of the fin (the curve) and a circumference arc. 
These curvature features are based on the determination of the center of the arc of 
circumference that better fits the polynomials at each side of the fin. Supposing that the 
circumference center is at (x c , y c ) and that n points are chosen along the curve, the center is 
calculated in such a way to minimize the cost of the function J(x c , y c ) defined by: 



J(x c .yc)=Zi i 2 -i i -i 



i=i 



where 



^-xJMYi-y,) 2 



By solving this equation, the following matrix equation can be found: 



AX=B 



This can yet be expanded to: 

2(x - xj 



2(y -yi) 
2^-yJ 



2( x n-i-x„) 2(y n _ 1 -y n ) 



(x 2 -x 2 ) (y 2 -y 2 ) 

(x^-x 2 ) (y 2 -y 2 ) 



.(x 2 _!-x n 2 ) (yL-y 2 ) 
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The Equation is solved applying a SSD approach. After obtaining the center (x C/ y c ), the 
Euclidean distances from it to each one of the n points of the considered fin side are 
calculated. The radius is given by the mean value of these distances. Fig. 11 shows two fins 
that differ in their radius sizes. This can be even visually observed mainly for the left side 
curves, the right one is smaller, so this is proven to be a relevant feature. 





* 


m 


" f *Xtt- 







Fig. 11. Two distinct fins that can be identified by the difference in the radius (see text for 
explanation). 

The discrepancy index is given by the sum of squared differences between the mean radius 
and the distance to each of n points considered. Note that for fin formats exactly as a 
circumference, this index would be zero. The two fins of Fig. 11 have indexes close to zero 
on the left side, besides different radius. Note that two fins with almost the same calculated 
radius may have different discrepancy indexes, as for the ones on right sides of Fig. 11. 



PeakX 


0.0958149 


PeakY 


0.835369 


Number of Holes 


NumHoles 1 


0.4 


NumHoles 2 


0.2 


Location 1 


Holesl 1 


0.129883 0.546875 


Location 2 


Holes 2 


0.640625 


Radius 1 


0.357855 


Radius 2 


0.436765 


Index 1 


0.034204 


Index 2 


0.010989 



Table 1. Features calculated for a given image of a dolphin. 



4.5. The feature extraction process for identification 

After all processing presented at previous sections, features for identification are extracted 
using the two polynomials. Between all measures (values) that can be used as features, we 
have found 16 relevant features: 

a) Two coordinates (x and y) of the peak; 

b) Number of holes/ cracks in each curve (from observations in the population, a 
number between and 4 is considered) normalized (that is, multiplied by 2/10); 
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c) The locations of holes/ cracks for each curve, at most 4, being given by the curve parameter value 
at the respective position (this parameter is already between and 1); 

d) Radius of curvature, corresponding to each parametric curve; 

e) Discrepancy indexes to the circumference for each curve; 

Table 1 shows an example of features extracted for a given animal image, which are already 
normalized. The number of holes (NumHoles = 0.4 and 0.2) are normalizations given by 
2(2/10) and 1(2/10), respectively, and two locations for Holesl and one for Holes2 are 
defined, the others are zero. Also note that the discrepancy index of the second side (Index2) 
is smaller. This indicates the shape of that side is closer to a circumference arc. 

5. Identification and recognition 

In the context of this work, identification means the determination of a specific animal 
instance by definition or observation of specific features from the images of it. That is, 
features extracted from the picture compared with a model (using a classifier) would give as 
result this specific animal. In a general sense, we say that a specific object can be identified 
by a matching of its specific features. Recognition means the current observation of a 
previously seen animal or group of animals. In this sense, recognition complains with a 
more general classification context, in which a given class can be recognized by way of 
matching some features of it. So in this work we might be more interested in identification 
than recognition, besides the last would also be required somewhere. 

5.1. Template matching 

Two techniques are used in this work for identification/ recognition. The first uses simple 
template matching to determine the distance of the feature vector extracted from a given 
image to a previously calculated feature vector, a model existing in an internal population 
table (like a long term memory). That is, for each vector in the table, the difference is 
calculated for all of its components, being given a score for it. At the end, the highest scored 
vector is assumed to be the winner, so the specific dolphin identified. Being fi a feature 
value, rrn a value of the corresponding feature in the model (table), n the number of features 
in the vector, the Equation for calculating the score for each pattern in the table is given by: 

s = 10-^yf.-m i 
n ti 

In the case of a total matching, the resulting score would be 10. Because all features are 
normalized, the summation would be up to n (number of features). In this work, number of 
features are 16). In the worst case (no matching at all), the score would be zero. Note that the 
above matching procedure ever produces a winner no matter its feature values are stored or not 
in the table. So, it would be hard to determine if a new individual is found by looking the results, 
in the case a minimum threshold for a positive identification is not set. Of course, by hand (visual 
comparison) this would be harder, or even impracticably. We thought in establishing a threshold 
in order to verify for new individuals, but that shows to be hard too. The problem here is how to 
define the most relevant features that may separate one individual to another. That varies from 
dolphin to dolphin. For example, the curvature of the fin may be the key for separating two 
dolphins, but the number of holes may be the key for other two. Then, we decided to find 
another, more efficient technique for identification. Besides, this technique initially allowed a 
qualitative analysis of the extracted features, verifying them as good features. 
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5.2. Using a backpropagation network 

In a previous work (Garcia, 2000) we used a multi-layer perceptron trained with the 
backpropagation algorithm (BPNN) (Braun, 1993). We decided to verify its application to this 
work case. So in a second implementation, the BPNN model is applied. Basically, the BPNN is 
a fully connected, feed-forward multi-layer network. That is, activation goes from the input to 
the output layer. Knowledge is codified at the synapses (links with weights), in each 
connection between the layers, at unities that are called knots or neurons. In the training phase, 
the synapse weights are adjusted by a process which has basically two steps (or phases). In the 
first step, called forward, the algorithm calculates the forward activation in the net using 
current weights. In the second step (the back propagation step), the error in the last layer is 
calculated as the difference between the desired and calculated outputs. This error is 
propagated to back, to intermediate layers which have their weights approximated corrected 
following an adjusting function. Then, another input is chosen and the forward step is done 
again for this new instance, and so on until the weights reach a stability state. Ideally, random 
instances are to be chosen at each step, and the net would be trained at infinity to produce 
exactly matches. Of course, a time limit is also set to stop it, but so after a good approximation 
is reached. After the net gets trained, if an input is presented to it, activation flows from the 
input to the output layer, which represents the correct answer to the input pattern. 




Fig. 12. Topology of backpropagation network used. 

The BPNN algorithm and its variations are currently well defined in the literature 
(Rumelhart, 1986; Werbos, 1988; Braun, 1993; Riedmiller, 1993} and implementations of it 
can be found in a series of high-level tools (as WEKA, Mathlab, etc). We implemented our 
own code, using C++ language. 

Understanding that the net must act as a (short) memory, learning automatically new 
features, we have proposed an extension in the BPNN original structure in such a way new 
instances of dolphins can be added without needs of user intervention. That is, the system 
would find itself a new dolphin instance by looking at the output layer results, acquire the 
new feature set for this new input, re-structure the net, and retrain it in order to conform to 
the new situation (producing as result a new dolphin added to the memory). We named this 
extension as a self-growing mechanism. 

Fig. 12 shows structure of the used in this work. It has one input for each feature above 
described (totaling 16 inputs). The number of nodes in the hidden layer changes 
according to the number of, actually known, instances of dolphins (self-growing). This 
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number was determined empirically, 1.5 x the number of nodes in the last layer produced 
good results. The number of nodes in the last layer changes dynamically. They are the 
number of dolphins already known. So a new output node is added for each new instance 
of dolphin detected in the environment. This is a very useful feature, avoiding human 
intervention for new instances. Note that each one of the extracted features (see Table 1) is 
normalized (values between and 1). This is due to imposed restrictions on the BPNN 
implementation, in order for the training procedure to converge faster. If an input feature 
vector representing an individual already in the memory is presented, the activation for 
the corresponding output would be ideally 1, and all the others 0. So the question is how 
to define a new individual, in order to implement the above mentioned self-growing 
mechanism. A simple threshold informing if an instance is new (or not) was initially set 
experimentally. For a given input, if its corresponding output activation does not reach 
the threshold, it could be considered as a possible new instance. In addition all the not 
activated outputs also must be very close to 0. In fact, this simple procedure showed some 
problems and we decided for another way, defining a weighting function, using the net 
errors. The novelty of an input vector is given by a function that considers the minimum 
and maximum error of the net, given in the training phase, and the current activations in 
the last layer. Considering o a as the value of the current most activated output, the current 
net error £, is given by £, = 2/2(2-o a + l/(n-l)Toi), where Oi are the values of the other n-1 
output nodes (i^a). Being ^ min , ^ max the minimum and maximum training errors, 
respectively, x a threshold, the value (true or false) of the novelty rj is given by: 

r, = ^> ((l-T) + (Un+^max)/2) 

Note that we still consider the activation threshold, which is adjusted empirically. This 
procedure was tested with other applications, and features, and proved empirically to be 
correct for getting new instances. Feed-forward activation (in training or matching) is 
calculated as: 

-2>u x 
1+e i=0 

j 

The following Equations are used for training the net: 

Am ]] (t + i)=£S ] ] +aAG7ij(t + l) 
where Oi is as defined above and 

fo^l-oJyj-Oj), Vje output 

lo J (l-o J E<W f Vjeothers 



*j=- 



6. Experimental results 

In order to test both identification algorithms and to prove the usefulness of the feature set 
proposed, we have performed several experiments. Basically, several images of different 
animal instances preferably in a regularized positioning are selected, the images passed by 
all the processes mentioned above, and the features calculated. These features are used 
either as the model or as input to both tools. In the following, we detail the experimentations 
performed and their results. 
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6.1 Template matching results 

For the first set of experiments (template matching), the table containing the feature set of 
selected animals is constructed in memory. Another image set, of dolphins already in memory, 
with poses varying a little or strongly in relation to the original one, is chosen from the database 
and presented to the system. By the results shown in Table 1, we can see that one of the animals 
is wrongly identified. That is, a feature set extracted from an image of a given animal present at 
the table was matched to another one. That is due to a certain degree of distortion of the animal 
pose in the presented image in relation to the original image pose, to image noise, to illumination 
bad conditions, or even to very similar features from a dolphin to another. 



Minimum score 


8,509124 


Maximum score 


9,914631 


Positive identification 


4 


False identification 


1 



Table 2. Images of known dolphins are presented for template matching. 

Next, in this experimentation set yet, another set of images of animals not previously in the 
table are presented. All of them have had wrong positive identification, of course, but, 
surprisingly, some of them even with high (good) scores (0.065% of total error). Table 3 
shows the scores for this experimentation. 



Minimum score 


7,192648 


Maximum score 


9,341214 


Positive identification 





False identification 


5 



Table 3. Results for images of dolphins not present in the Table, using template matching. 



6.2. Experiments with the backpropagation net 

For experimentations using the BPNN, the net is first trained with a current set of features. In 
our work, it is initialized with no dolphins. For implementation purposes, the net is initialized 
with two output nodes, the first for all input feature values equals to and the second for all 
input values equals to 1. So, in practice, there exist two virtual dolphins, only for the 
initialization of the memory. As all of the real images have not all the features in nor in 1, 
exclusively, these virtual dolphins (outputs) will not influence. So, at the very first time, the 
network is trained in this configuration. The activation threshold is initially set to 0.90. 
The first image is presented, its features extracted, and matched by the network. The 
memory presents some activation for the two existing nodes, under the threshold for both. 
Novelty given above is set to true meaning that a new instance of dolphin is discovered. The 
self-growing mechanism inserts the new set of features in the net, and retrains it, now 
containing a real dolphin features on it, besides the two initial nodes. Next, other set of 
features of a different dolphin is presented. The above procedure is repeated, and so on, 
until features of 8 different dolphins could be inserted in the net. The ninths has had 
activation over the threshold that was initially too low but was wrongly identified. So the 
threshold might be set upper, in order to be more selective or another way (function) to 
discover the new dolphins might has to be found. The first option is chosen, but we stop 
here this experiment in order to try with same pictures already in the memory, and to other 
cases of same dolphins with modified poses. So, as an initial result, in all 8 cases (not 
previously existing individuals) the activation is all under the threshold, so they could be 
inserted in the net as new instances. At the final configuration of the net (10 outputs), it took 
500 epochs for training in less than 1 second in a Pentium 4 processor, 3.0 GHz. 
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Activated 


0.95 


0.04 


0.01 


0.00 


0.01 


0.00 


0.02 


0.00 


Desired 


1.00 


0.00 


0.00 


0.00 


0.00 


0.00 


0.00 


0.00 
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0.03 
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0.04 
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0.01 


0.00 


0.00 


0.01 
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1.00 


0.00 
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0.00 


0.00 


0.00 


0.00 
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0.04 


0.94 
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0.00 


0.00 


0.02 


0.03 
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0.00 


0.00 


1.00 


0.00 


0.00 


0.00 


0.00 


0.00 
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1.00 


0.00 


0.00 


0.00 


Activated 


0.01 


0.00 


0.00 


0.00 
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0.03 
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0.00 
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0.00 
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0.00 


0.00 


0.00 


0.00 


0.00 


0.00 


1.00 


0.00 


Activated 


0.00 


0.04 


0.03 


0.02 


0.01 


0.03 


0.00 


0.96 


Desired 


0.00 


0.00 


0.00 


0.00 


0.00 


0.00 


0.00 


1.00 



Table. 4. Output of the net (activated and desired). 



In order to verify robustness of the training process, the same image set is then presented to 
the net. All of them are positively identified, as expected. The training and presented set are 
exactly the same, this is to test the net trustworthiness. Table. 4 shows the activation in the 
output layer nodes and their desirable activations (as same images are used). The maximum 
error is 0.076041 (ideally would be zero), minimum error is zero and the mean error is 
0.015358 (ideally would be zero). This demonstrates that the training algorithm works on the 
desired way, converging to an adequate training. 

Next, other 4 images of instances already present in the Table (1, 3, 5, 7), with little 
modification in pose are presented to the net. They are all positively identified, and, as can 
be seen in Table 4, the activation to its corresponding model is over 0.85 in all of them. The 
next experiment presents results for 3 images with strongly modified positioning in relation 
to the model. This means poses are rotated almost 30 degrees in relation to the models. 
There is one case with activation under 0.85, and with other nodes more activated than 
expected (see Table 6). Cases like this one could be considered as confusing, making the self- 
growing mechanism act, wrongly.) 
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Table. 5. Activation given by the last layer of the net for dolphins with little modification in 
pose. 
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0.00 


1.00 
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Table. 6. Activation given by last layer of net, dolphins with strongly modified poses up to 
30 degrees rotated. 

In relation to the training time, the graphic illustrated in Fig. 13 shows processing time in 
miliseconds versus number of dolphins (that is, number of nodes of last layer plus two) in the 
net. One can note that for 10 dolphins the net takes some 200 ms. The graphic illustrated at Fig. 
14 shows number of epochs versus number of dolphins. We note that the input layer is 
composed of 16 nodes, a 16 degree dimensional feature vector space. With 28 dolphins, the 
hidden layer has 70 nodes. The graphics show an apparently exponential function. This is not 
a problem since the number of dolphins in a family should not be more than 50. Yet, in a 
practical situation, the system is trained only when a new animal is found. That can take some 
time without problems since the system is trained of -line. In this case, the weights can be saved 
to be used in the time it becomes necessary, not interfering in the system performance. 




5 

Number of dolphins 

Fig. 13. Time versus number of dolphins in the net. 




10 20 

Number of dolphins 



Fig. 14. Epochs versus number of dolphins. 
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For on-line situations or for situations in which the net should be trained fast, other 
approaches can be used. As for example, one solution is to increase the input space 
state that each node of the output can reach. One way to do that is to set each output 
for more than one instance, dividing the range of each output from zero to one in 
more than one value depending on the number of dolphins. For example, instead of 
one output for each dolphin, a binary notation can be used, addressing two dolphins. 
So, with 20 outputs, a number of 2 20 of dolphins can be addressed by the net. For 50 
dolphins, 6 nodes on the output layer are enough, instead of the 50 in the current 
implementation. Of course, training time may not decrease so fast as the number of 
nodes, but for sure it will be faster due to necessity of less computation for feed- 
forwarding the net and back propagating the error. By comparing the above results 
for both classifiers, the net and the straight-forward method using template matching, 
we could see that the net, besides slower, performed better producing activations 
with more reliable values. Yet, it is easier to deal with, inserting features for new 
instances in automatic way. 

7. Conclusion and future research 

In this work, we have proposed a complete system for identification of dolphins. The 
system involves image acquisition, pre-processing, feature extraction, and 
identification. The novelties of the system are mainly in the proposed methodology for 
pre-processing the images based on KLT for extraction of the dorsal fin, the new 
feature set proposed for being used as input to the classifiers, and the self-growing 
mechanism itself. With that, the system does not need user intervention for training its 
memory, neither for identifying new subjects in the universe of animals. The system 
has proceeded as expected in all experiments. 

Of course, other alternatives for the classifier can be tested in the following, for 
example, Bayesian nets, self organizing maps, or radial basis functions. Self organizing 
maps has the property of separating the input space in clusters. We do not know if this 
would be a good strategy, it has to be tested, perhaps for separation between families. 
It is important to remark that the feature set used can be enhanced. A good set must 
consider texture, besides shape features. In order to use texture, one must use an 
approach for avoiding water effects in the image illumination. There are some 
restrictions to be observed in the feature extraction. For example, in the case of 
substantial rotation, close to 90 degrees, the holes or cracks may not be visible. Holes 
may not be detected, prejudicing the system performance. Of course, a visual 
procedure in the considered picture would not produce any good result too, as in this 
case the fin may become visible from the front or from the back of the dolphin. So, 
besides the necessity of such improvements, we enhance the importance of the features 
proposed, which has produced good results. 

The BP net used showed to be useful in the insertion of new individuals in the long term memory. 
Yet, some strategy can be developed in order for the system to learn what features are the best to 
segregate individuals from a given group in a more precise way. If the group changes, the fins 
characteristics may also change. In this way, a smaller feature set can be used, diminishing training 
time and growing in efficiency. Also, using weights in the features is another strategy to be tried. 
The idea is to determine the most relevant features, for specie specifics. A stochastic inference 
approach can be tried, as future work, in this track. 
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1. Introduction 

Landmines are prominent weapon and they are so effective, yet so cheap, and easy to make 
and lay. A landmine is a type of self-contained explosive device, which is placed onto or into 
the ground to constitute a minefield, and it is designed to destroy or damage equipment or 
personnel. A mine is detonated by the action of its target (a vehicle, a person, an animal, 
etc.), the passage of time, or controlled means. A minefield is an area of ground containing 
mines laid with or without a pattern. They are the most effective means of reinforcing the 
terrain to stop, slow, or force the enemy to pass into areas where he can be killed. There are 
two types of land-based mines: anti-tank (AT) and anti-personnel (AP). The production 
costs of AP mines are roughly between 3 and 30 US$. AP mines can kill or incapacitate their 
victims. The removal and destruction of all forms of dangerous battlefield debris, 
particularly landmines and other unexploded ordnance (UXO), are vital prerequisites for 
any region to recover from the aftermath of a war. Additional major effect of mines is to 
deny access to land and its resources, causing deprivation and social problems among the 
affected populations. Besides this, the medical, social, economic, and environmental 
consequences are immense (O'Malley, 1993; Blagden, 1993; Physicians for Human Rights, 
1993; US Department of State, 1994; King, 1997; ICRC, 1998). The international Committee of 
the Red Cross (ICRC) estimates that the casualty rate from mines currently exceeds 26,000 
persons every year. It is estimated that 800 persons are killed and 1,200 maimed each month 
by landmines around the world (ICRC, 1996a; ICRC, 1996b; ICRC, 1998). The primary 
victims are unarmed civilians and among them children are particularly affected. 
Worldwide there are some 300,000-400,000 landmine survivors. Survivors face terrible 
physical, psychological and socio-economic difficulties. The direct cost of medical treatment 
and rehabilitation exceeds US$750 million. This figure is very small compared to the 
projected cost of clearing the existing mines. The current cost rate of clearing one mine is 
ranging between 300-1000 US$ per mine (depending on the mine infected area and the 
number of false alarms). United Nation Department of Human Affairs (UNDHA) assesses 
that there are more than 100 million mines that are scattered across the world and pose 
significant hazards in more than 68 countries that need to be cleared (O'Malley, 1993; 
Blagden, 1993; Physicians for Human Rights, 1993; US Department of State, 1994; King, 1997; 
Habib, 2002b). Currently, there are 2 to 5 millions of new mines continuing to be laid every 
year. Additional stockpiles exceeding 100 million mines are held in over 100 nations, and 50 
of these nations still producing a further 5 million new mines every year. The rate of 
clearance is far slower. There exists about 2000 types of mines around the world; among 
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these, there are more than 650 types of AP mines. What happens when a landmine explodes 
is also variable. A number of sources, such as pressure, movement, sound, magnetism, and 
vibration can trigger a landmine. AP mines commonly use the pressure of a person's foot as 
a triggering means, but tripwires are also frequently employed. Most AP mines can be 
classified into one of the following four categories: blast, fragmentation, directional, and 
bounding devices. These mines range from very simple devices to high technology 
(O'Malley, 1993; US Department of State, 1994). Some types of modern mines are designed 
to self-destruct, or chemically render themselves inert after a period of weeks or months to 
reduce the likelihood of civilian casualties at the conflict's end. Conventional landmines 
around the world do not have self-destructive mechanisms and they stay active for long 
time. Modern landmines are fabricated from sophisticated non-metallic materials. New, 
smaller, lightweight, more lethal mines are now providing the capability for rapid 
emplacement of self-destructing AT and AP minefields by a variety of delivery modes. 
These modes range from manual emplacement to launchers on vehicles and both rotary and 
fixed-wing aircraft. Even more radical changes are coming in mines that are capable of 
sensing the direction and type of threat. These mines will also be able to be turned on and 
off, employing their own electronic countermeasures to ensure survivability against enemy 
countermine operations. Although demining has been given top priority, currently mine's 
clearing operation is a labor-intensive, slow, very dangerous, expensive, and low technology 
operation. Landmines are usually simple devices, readily manufactured anywhere, easy to 
lay and yet so difficult and dangerous to find and destroy. They are harmful because of their 
unknown positions and often difficult to detect. The fundamental goals of humanitarian 
landmine clearance is to detect and clear mines from infected areas efficiently, reliably and 
as safely and as rapidly as possible while keeping cost to the minimum, in order to make 
these areas economically viable and usable for the development without fear. 
Applying technology to humanitarian demining is a stimulating objective. Detecting and 
removing AP mines seems to be a perfect application for robots. However, this need to have a 
good understanding of the problem and a careful analysis must filter the goals in order to 
avoid deception and increase the possibility of achieving results (Nicoud, 1996). Mechanized 
and robotized solutions properly sized with suitable modularized mechanized structure and 
well adapted to local conditions of minefields can greatly improve the safety of personnel as 
well as work efficiency and flexibility. Such intelligent and flexible machines can speed the 
clearance process when used in combination with handheld mine detection tools. They may 
also be useful in quickly verifying that an area is clear of landmines so that manual cleaners 
can concentrate on those areas that are most likely to be infested. In addition, solving this 
problem presents great challenges in robotic mechanics and mobility, sensors, sensor 
integration and sensor fusion, autonomous or semi autonomous navigation, and machine 
intelligence. Furthermore, the use of many robots working and coordinating their movement 
will improve the productivity of the overall mine detection process with team cooperation and 
coordination. A good deal of research and development has gone into mechanical mine 
clearance (mostly military equipment), in order to destroy mines quickly, and to avoid the 
necessity of deminers making physical contact with the mines at all. Almost no equipment has 
been developed specifically to fulfill humanitarian mine clearance objectives and for this, there 
is no form of any available mechanical mine clearance technologies that can give the high 
clearance ratio to help achieving humanitarian mine clearance standards effectively while 
minimizing the environmental impact. Greater resources need to be devoted to demining both 
to immediate clearance and to the development of innovated detection and clearance 
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equipment and technologies. This chapter introduces the problem of mines and its impact. It 
also, focuses on the aspects of demining, the requirements and the difficulties facing it. Then, 
the chapter evaluates the available mine clearance technologies along with their limitations 
and discusses the development efforts to automate tasks related to demining process wherever 
possible through mechanization and robotization. It aims to evaluate current humanitarian 
demining situations and technologies for the purpose to improve existing technologies and 
develop an innovative one. In addition, it introduces solutions and priorities beside the 
requirements in terms of technical features and design capabilities of a mobile platform that 
can accelerate the demining process, preserve the life of the mine clearing personnel and 
enhance safety, and achieve cost effective measures. 

2. Service Robots 

Between the 60s and end of 80s, most robot applications were related to industries and 
manufacturing and these robots were called industrial robots that were mainly intended for 
rationalizing production at a manufacturing site. A robot is usually an extremely flexible and 
complex machine, which integrates science and engineering. Each technology used in the 
robotic system has its own challenges to offer. The opportunity for robotics to help humanity 
arises when there are not enough skilled people available to do certain tasks at a reasonable 
price, like elder care. Much thought has been put into development of robotic helpers for the 
infirmed and elderly. Advances in micro-technology, microprocessors, sensor technology, 
smart materials, signal processing and computing technologies, information and 
communication technologies, navigation technology, and biological inspiration in learning and 
decision-making capabilities have led to breakthrough in the invention of a new generation of 
robots called service robots. Service robot is a generic term covering all robots that are not 
intended for industrial use, i.e., perform services useful to the well being of humans, and other 
equipment (maintenance, repair, cleaning etc.), and are not intended for rationalizing 
production. The development and operation of service robots provide invaluable experience 
as they form an intermediate stage in the evolution from the industrial robot to the personal 
robot, which is recognized as an important application area for the near future. The new types 
of robots aim to achieve high level of intelligence, functionality, flexibility, adaptability, 
mobility, intractability, and efficiency to perform wide range of work in complex and 
hazardous environment, and to provide and perform services of various kinds to human users 
and society. Crucial prerequisites for performing services are safety, mobility, and autonomy 
supported by strong sensory perception. Such robots should be good at what they can do, and 
have the ability to work at a larger degree of unstructured environments. In addition, human- 
robot interaction plays a crucial role in the evolving market for intelligent personal robots. 
Service robots are manipulative and dexterous, and have the capability to interact, perform 
tasks autonomously/ semi autonomously (multi modes operation), and they are portable. 
Three classes of service robots can be distinguished, the first being robots to replace humans at 
work in dirty, hazardous and tedious operations, such as working under high temperature, in a 
radioactive environment, in a vacuum, underwater, fire fighting, space, demining, military, 
construction, cleaning etc. The second class includes robots that operate with human beings to 
alleviate incommodity or to increase comfort, such as, entertainment, rehabilitation, assist the 
elderly and severely disabled, housekeeping, etc. The third class includes robots that operate on 
human being, such as medical robots mainly for surgery, treatment and diagnosis. 
Service robots with their free navigation capability target a wide range of applications, such 
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as agriculture & harvesting, healthcare/ rehabilitation, cleaning (house, public, industry), 
construction, humanitarian demining, entertainment, fire fighting, hobby/ leisure, 
hotel/ restaurant, marketing, food industry, medical, mining, surveillance, inspection and 
maintenance, search & rescue, guides & office, nuclear power, transport, refilling & 
refueling, hazardous environments, military, sporting, space, underwater, etc. Such robots 
aim to offer a useful service with reasonable cost compared to expected duties. 

3. Humanitarian demining 

Humanitarian demining scenarios differ from military ones in many respects. The 
objectives and philosophy are different. Solutions developed for the military are generally 
not suitable for humanitarian demining. Humanitarian demining is a critical first step for 
reconstruction of post-conflict countries and its goal is the total clearance of the land from 
all types of mines and UXOs. It requires that all mines in populated areas and other 
infrastructures are located, uncovered and removed or destroyed. It is carried out in a post- 
conflict context, and its objective to decontaminate mine-infected areas and to make sure 
that there is not a mine being left in the ground. The important outcome of humanitarian 
demining is to make land safer for daily living and restoration to what it was prior to the 
hostilities. Also, it is allowing people to use their land without fear; allowing refugees to 
return home, schools to be reopened, land to be used for farming and critical infrastructure 
to be rebuilt (ESPIRIT HPCN, 1997; Bruschini et al, 1999; Habib, 2002b; Goose, 2004). The 
standard to which clearance must be achieved is extremely high as there is a need to have at least 
99.6% (the standard required by UNDHA) successful detection and removal rate (Blagden, 
1993), and a 100% to a certain depth according to International Mine Action Standards (IMAS). 
The amount of time it takes to clear an area is less important than the safety of the clearance 
personnel and the reliability and accuracy of the demining process. Safety is of utmost importance, 
and casualties are unacceptable. Any system to be developed should compliment this effort, not to 
hamper it or simply move the problem elsewhere. The risks to those carrying out the task must 
also be maintained at a lower level than might be acceptable in a military situation. Another 
consideration by humanitarian demining is the use of land for development, i.e., there is a need to 
reduce the environmental impact that may results from the demining operation. The currently 
available technologies are not suited to achieve these objectives of humanitarian demining. Until 
now, detection and clearance in humanitarian demining very often relies on manual methods as 
primary procedure. The problem resides primarily in the detection phase first, and then how to 
increase productivity by speeding up demining process reliably and safely. Technology has 
become the solution to many long-standing problems, and while current technology may be 
effective, it is far too limited to address fully the huge, complex and difficult landmine problem 
facing the world. 

4. Humanitarian Mine Clearance and Difficulties 

Humanitarian demining requires that the entire land area to be free of mines and hence the 
need to detect, locates, and removes reliably every single mine, and UXO from a targeted 
ground. The development of new demining technologies is difficult because of the 
tremendous diversity of terrains and environmental conditions in which mines are laid and 
because of the wide variety of landmines. There is wide range of terrains (rocky, rolling, 
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flat, desert, beaches, hillside, muddy, river, canal bank, forest, trench, etc.) whereas mines 
are often laid. The environmental conditions may cover weather (hot, humid, rainy, cold, 
windy), the density of vegetation (heavy, medium, small, none), and type of soil (soft, sand, 
cultivated, hard clay, covered by snow, covered with water). In addition, residential, 
industrial and agriculture area, each has its own features and needs to be considered. 
Landmines are many in terms of type and size. The size ranges from small enough to fit 
into a hand of a child, and weighting as little as 50 grams, to large antitank mines. AP 
mines come in all shapes and colors are made from a variety of materials, metallic and 
nonmetallic. Metal detector works well with metal cased mines, but metal in modern mines 
has been increasingly replaced by plastic and wood. New mines will soon be undetectable 
by their metallic content. AP mines can be laid anywhere and can be set off in a number of 
ways because the activation mechanisms available for these mines are not the same. 
Activation methods can be classified into three categories, pressure, electronic, and 
command detonation (remote control). Mines may have been in place for many years, they 
might be corroded, waterlogged, impregnated with mud or dirt, and can behave quite 
unpredictable. Some mines were buried too deep to stop more organized forces finding 
them with metal detectors. Deeper mines may not detonate when the ground is hard, but 
later rain may soften the ground to the point where even a child's footstep will set them off. 
Trip-wires may be caught up in overgrown bushes, grass or roots. In addition, there is no 
accurate estimate on the size of the contaminated land and the number of mines laid in it. 
The diversity of the mine threat points out to the need for different types of sensors and 
equipment to detect and neutralize landmines. The requirements to develop equipment for 
use by deminers with different training levels, cultures, and education levels greatly add to 
the challenge. The solution to this problem is very difficult and challenging one from a 
scientific and technical point of view because, given the nature of landmines and the 
requirements of humanitarian demining, any instrument must be 100% reliable for the 
safety of the operators and the people whom will use the land (Blagden, 1993; Habib 2002b). 
Developing new technologies is critical to the success of the efforts intended to reduce this 
threat. 

5. Humanitarian Mine Clearance Process and Needs 

Land mine clearance process can be divided into the following essential five phases (Habib, 
2002b). 

5.1 Locating and identifying minefields for the purpose to map them 

Demining is very costly and searching unmined area is adding extra high cost. Therefore, it 
is important first to identify and mark what areas are mined using different approaches 
through survey and remote sensing along with other techniques. A clearance priority rating 
should be given to each suspected mined area to reflect the urgency for clearance while 
considering social and economical factors. 

5.2 Preparing the minefield for the clearance operation 

The normal obstacles facing deminers that considerably slow down the operation are 
surface vegetation and subsurface metal contamination. Currently, as much as 70% of 
deminer's time spent in checking for tripwires and cutting back vegetation through the 
demining process. 
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5.3 Locating and Marking Individual Mines 

Different ways are used to detect individual mines such as manual probing, dogs, metal 
detectors, thermal imaging, electromagnetic technologies, nuclear technologies, chemical 
techniques, biosensors, etc. Generally, such methods are limited by soil conditions, 
vegetation, mine size and composition, soil minerals, burial depth, and grazing angle. Some 
of these technologies are still under development and experimentation. Cost and size is 
among the difficulties in deploying advanced and high-tech sensors. 

5.4 Removing the threat of the detected mines 

Once a mine has been located and marked, it must be neutralized to render its harmless. 
Detection and neutralization of landmines are difficult tasks, as there are a lot of different 
types of mines. These differences can be situated on different levels, the geometry of the 
mines with the materials used for the casing, or the fusing of the mine ranging from simple 
pressure fuses to more sophisticated ones as magnetic, acoustic, seismic, etc. The most 
difficult mines to detect are the pressure mines. Among the available demining methods, 
there exist different means of mine neutralization. 

5.5 Quality Control Measures. 

There is a need to verify that the required demining standards have been achieved, i.e., to 
assure with high level of confidence that the cleared area is free from mines and UXOs. 
The weather and environmental conditions along with ground surface movement should be 
considered as factors to schedule demining activities, due to its impact on deminer's and 
equipment reliable performance. 

6. Solutions and Priorities 

Current demining technology is slow, expensive, and dangerous. The current rate of humanitarian 
mine clearing is about 100 thousand per year. It is estimated that the current demining rate is 
about 10-20 times slower than the laying rate, i.e., for every mine cleared 10-20 mines are laid. 
Therefore, to stabilize the mine situation, it is necessary to increase the current capability of mine 
clearance by 10-20 times. Hence, it becomes urgent to develop detection (individual mine, and 
area mine detection), identification and removal technologies and techniques to increase the 
efficiency of demining operations by several orders of magnitude to achieve a substantial 
reduction to the threat of AP mines within a reasonable timeframe and at an affordable cost. 
The priorities for research and development in the field of humanitarian demining require 
strategies that should start with the following needs: 

a) To develop reliable and accurate techniques that can enhance the performance of the 
demining process and allow efficient area detection of minefields. There is an urgent 
need to recognize and reliably locate minefields and isolate them by defining proper 
signs and limits to make the public aware, and to avoid further accidents. 

b) To have quality-training programs that fit the needs of local environment. Such training 
programs need to integrate cultural, environmental and operational considerations 
when developed. 

c) To enhance the safety of deminers by providing them with suitable protective clothing 
and equipment and by isolating them from direct physical contact with the mines and 
UXOs as possible. 
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d) To enhance the performance of the sensors and the deminers. To achieve this, there is a 
need to develop efficient techniques for sensor integration (array of homogeneous or 
heterogeneous sensors) with advance level of data fusion and signal processing 
algorithms that can confirm the detection and lead to the identification of mine 
parameters needed for the next actions. 

e) To develop a portable, reliable and easy to use handheld approach to sensor movement 
that is still required in difficult and physically constraint environments (woods, uneven 
terrain, residential, etc.) although such approach is slow and hazardous for the 
individuals. Therefore, the sensors can be integrated with vehicle-based platforms to 
support automatic mine clearance in open areas. 

f) To use information and communication technologies aiming to enhance contact, 
experience exchange, research, planning and to share results and data among all parties 
and personnel within the demining community. 

g) To mechanize vegetation cutting. However, it would be better to find a technology that 
can detect and mark mines without having to cut vegetation. 

h) To increase mine clearance daily performance by improving productivity, accuracy, and 
increase safety of demining personnel. There is a need to have an automatic means of 
moving the portable mine detection device as it searches for landmines. Hence, it is 
important to automate/ mechanize the detection and removal processes of mines, and to 
improve the safety of the deminers through the use of efficient, reliable and cost effective 
humanitarian mine action equipment (such as robots, flexible mechanisms, etc.), that 
have minimum environmental impact. It is necessary to have a robot with efficient 
surface locomotion concept and mobility that is well adapted to unstructured 
environment. The design should integrate proper balance between maneuverability, 
stability, speed, and the ability to overcome obstacles. Such robots, should have 
decision-making capability to locate, mark or neutralize individual mines precisely, 
i) To have efficient quality control assurance methods that is reliable and accurate in 

ensuring that an area is clear of mines. 
In order to approach proper and practical solutions for the problem, there is a need for 
the scientists in each discipline and deminers to share their knowledge and the results 
of their experience and experiments in order to design and test viable solutions for 
humanitarian demining. 

The challenges associated with configuring humanitarian demining equipments are many. 
Technologies to be developed should take into account the facts that many of the demining 
operators will have had minimal formal education and that the countries where the 
equipment is to be used have poor technological infrastructure for equipment maintenance, 
operation, and deployment. The resultant system must be inexpensive and easy to use with 
minimal training by locals. In addition, the equipment must be flexible and modular to 
address a variety of clearance tasks and for case-by-case scenarios. Furthermore, the 
logistical support of the equipment must be consistent with third world countries. 

7. Mine Detection and Sensing Technologies 

Mine detection represents the slowest and the most important step within the demining 
process, and the quality of mine detector affects the efficiency and safety of this process. 
Mine detection targets are to achieve a high probability of detection rate while maintaining 
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low probability of false alarm. The probability of false alarm rate is directly proportional to 
the time and cost of demining by a large factor. Hence, it is important to develop more 
effective detection technology that speed up the detection process, maximize detection 
reliability and accuracy, reduce false alarm rate, improve the ability to positively 
discriminate landmines from other buried objects and metallic debris, and enhance safety 
and protection for deminers. In addition, there is a need to have simple, flexible and friendly 
user interaction that allows safe operation without the need for extensive training. Such 
approach needs to incorporate the strength of sensing technologies with efficient 
mathematical, theoretic approaches, and techniques for analyzing complex incoming signals 
from mine detectors to improve mine detectability. This leads to maximize the performance 
of the equipment through the optimization of signal processing and operational procedures. 
Furthermore, careful study of the limitations of any tool with regard to the location, 
environment, and soil composition is critically important beside preparing the required 
operational and maintenance skills. Keeping in mind that not all high-tech solutions may be 
workable in different soil and environmental conditions. The detection technologies are 
presently in varying stages of development. Each has its own strength and weaknesses. The 
development phase of such new technologies requires a well-established set of testing 
facilities at the laboratory level that carried out in conditions closely follow those of the mine 
affected area, and at the real site. This should be followed by having extensive field trails in 
real scenarios to validate the new technologies under actual field conditions for the purpose 
to specify benefits and limitations of different methods. The work must be performed in 
close cooperation with end-users of the equipment and real deminers should carry out the 
test at a real site, in order to ensure that the developments are consistent with the practical 
operational procedures in the context of humanitarian demining, and that it is fulfilling user 
requirements. In addition, there is a need to have reliable process of global standard for 
assessing the availability, suitability, and affordability of technology with enabling 
technology represented by common information tools that enable these assessments and 
evaluations. The benchmarking is going to enhance the performance levels that enable the 
development of reliable and accurate equipment, systems and algorithms. 
Methods of detecting mines vary from, simple in technology but exhaustive searching by humans 
using some combination of metal detectors and manual probing, to a variety of high biological 
and electronic technologies. The effectiveness of metal detectors can be inhibited by mines with 
extremely low metal content or by soils with high ferrous content and hence other detection 
techniques have been and are being investigated. Another technique that is widely used is the 
direct detection of explosive material by smell using a dog (Sieber, 1995). Trained dogs are the best 
known explosive detectors but they need excessive training and inherently unreliable because 
they are greatly impeded by windy conditions, and have only 50-60% accuracy. 
New technologies are being investigated to improve the reliability and speedup the 
detection operation, some of these technologies are: Electromagnetic Induction metal 
detectors (EMI), Infrared Imaging, Ground Penetrating Radar (GPR), Acoustics, Acoustic 
Imaging, Thermal Neutron Activation (TNA), Photoacoustic Spectroscopy, Nuclear 
Quadrupole Resonance (NQR), X-ray tomography, Neutron back-scattering, Biosensors, 
Commercial sniffers, etc. (Healy & Webber, 1993; Van Westen, 1993; Hewish & Ness, 1995; 
Sieber, 1995; McFee, 1996; Cain & Meidinger, 1996; Habib, 2001a). 

Currently, there is no single sensor technology that has the capability to attain good levels of 
detection for the available AP mines while having a low false alarm rate under various types 
of soil, different weather, all types of mines, and facing many types of false targets. If one 
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sensor can detect a mine with a certain success rate coupled with a certain probability of 
generating a false alarm, could two sensors working together do a better job? The idea of 
developing multi sensor solutions involving two or more sensors coupled to computer 
based decision support systems with advanced signal processing techniques is attractive 
and is advocated by many as a fruitful line of development. Hence, there is a need to use 
complementary sensor technologies and to do an appropriate sensor data fusion. The 
ultimate purpose is to have a system that improves detection, validation and recognition of 
buried items for the purpose to reduce false alarm rates and to overcome current landmine 
detection limitations. A promising solution will be to apply fusion of sensory information on 
various sensor outputs through the use of advanced signal processing techniques, by 
integrating different sensor technologies reacting to different physical characteristics of 
buried objects. Here two main steps can be distinguished: first the presence of an object 
must be detected, and secondly this object has to be identified as being a mine or other. 
Critical to demining is the ability to distinguish fragments or stones from the target material 
in real time. Studying the characteristics of the data stream of each sensor separately is 
essential in order to extract those properties that allow discrimination between these two 
categories. There is a need to develop or enhance the currently available sensors and to use 
complementary sensor technologies and have appropriate sensor data fusion techniques. 
Sensor fusion using soft computing methods such as fuzzy logic, neural networks and 
rough set theory must be further explored and computationally inexpensive methods of 
combining sensory data must be designed. These methods should also have the capability to 
assess the quality of the mined area once the mines have been cleared. 

8. Humanitarian Demining Techniques 

The initial responses to the mine problem were essentially military in their structure and 
philosophy. The currently used demining methods are not safe for both, those clearing the 
mines, and those who must thereafter occupy the land that has been cleared. The methods 
are neither cost effective nor efficient. 

Mine clearance itself can be accomplished through different methods with varying levels of 
technology, but the most laborious way is still the most reliable. Currently, almost all 
humanitarian mine clearance is still performed by hand clearance method that uses 
'prodding 7 or 'probing'. Prodder consists of 30 cm long prod that deminer inserts into the 
soil at a shallow angle (approximately 30 degrees). It is the procedure where mines are 
manually detected and destroyed/ neutralized by a human deminer. Manual probing is 
slow, labor intensive and extremely dangerous and stressful process. 

Mechanical approaches rely on the use of motorized mine-clearers in which their design is 
influenced by the military demining requirements. Machines destroy or activate mines 
mechanically by hitting or milling the ground. A number of mechanical mine clearing 
machines have been constructed or adapted from military vehicles, armored vehicles, or 
commercially available agriculture vehicles of the same or similar type, with same or 
reduced size (Habib, 2001b). Mechanical mine clearance systems aim to unearth mines or 
force them to explode under the pressure of heavy machinery and associated tools. 
Mechanical clearance can cover more ground in less time than manual deminers and mine 
detection dogs. They are mostly appropriate in large and wide areas without dense 
vegetation or steep grades. In small paths or thick bush, such machines simply cannot 
maneuver. Thus, mechanical mine clearance is particularly suited for roads, and favorable 
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terrain such as flat, and sandy areas. Large mechanical systems, in particular the flail and 
tiller machines are expensive and requires substantial investments, not only for machine 
costs but also for logistics and maintenance, and actually can only be employed on a fraction 
of the total mined areas. The logistical problems associated with transporting heavy 
machinery to remote areas is critical in countries with little infrastructure and resources. 
Despite these advances, mechanical clearance equipment is of limited capability and it is 
unlikely, at least in the near term, to meet the minimum 99.6% effectiveness criteria and 
safety standards required by humanitarian demining. With this technique, machines often 
do not destroy all mines in a contaminated area, and AP mines may be pushed on side or 
buried deeper or partly damaged making them more dangerous. However, to achieve high 
clearance rate, the most effective way is to use these machines in conjunction with dog 
teams and/ or manual clearance team, which double check an area for remaining mines. 
Mechanical clearance equipment is expensive and it cannot be used on roadsides, steep hills, 
around large trees, inside a residential area, soft terrain, heavy vegetation or rocky terrain. 
Mobility and maneuverability where wheeled vehicles cannot travel efficiently on anything 
other than flat surfaces, tracked vehicles cannot travel in areas with steep vertical walls, 
machines in general cannot climb undefined obstacles, and machines cannot in general 
deform to get through narrow entrances. Also, it is important for such machines to work in a 
wide range of operational conditions such as, temperature and humidity and the need for 
protection against dust for engine and crew. In addition, mechanical clearance has its own 
environmental impact such as erosion and soil pollution. However, a single mechanical 
mine clearance unit can work faster than a thousand deminers over flat fields. Thus they are 
cost effective in limited circumstances. 

A number of mechanical mine clearing machines have been tested during the past. The general 
trend goes from " mechanical demining" towards " mechanically assisted demining", adaptable 
to local circumstances. Some examples of mechanical clearance equipment include but not 
limited, Vegetation cutters, Flails and Light-Flails, Panther mine clearing vehicle, Armored 
bulldozer, Ploughs and the rake plough, the M2 Surface "V" mine plow, Earth tillers, Mine 
sifter, Armored wheel shovel, Mine clearing cultivator, Floating mine blade, Rollers, Mine- 
proof vehicles, Swedish Mine Fighter (SMF), Armored road grader, etc. (US Department of 
Defense, 1999; Humanitarian Mine Action Equipment Catalogue, 1999; Department of 
Defense, 2202; Habib, 2002a; Geneva Centre for Humanitarian Demining, 2006). 
There is an urgent need to speed up the development to have compact and portable, low 
cost, technically feasible, fast response, safe, accurate, reliable, and easy to operate mine 
detector systems with flexible mobile platforms that can be reliably used to detect all types 
of available landmines and support fast and wide area coverage. Appropriate mine 
clearance technologies are those inexpensive, rugged, and reliable technical products, 
processes and techniques that are developed within, or should be transferred for use in 
mine-affected areas. They are also technologies that are cheap enough to be purchased 
within the regional economy and simple enough to be made and maintained in a small 
workshop. We should favor technologies that can be manufactured in mined countries; 
technologies that are transferable, and which provide employment and economic 
infrastructure where it is most urgently required. 

In addition, vegetation represents a large problem facing demining (mainly in tropical 
countries) and often poses major difficulties to the demining efforts. The vegetation removal 
can take up a substantial fraction of the time and for this there is a need to properly 
mechanized vegetation cutting and removal. These machines should be designed to cut 
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down on the time required for demining. In their simplest form, vegetation cutters consist of 
adequately modified commercial devices (e.g. agricultural tractors with hedge cutters or 
excavators). There is an urgent need for effective vegetation clearance technology and 
techniques that avoid detonating mines. Cost effective and efficient clearance techniques for 
clearing both landmines and vegetation have been identified as a significant need by the 
demining community 

9. Robotics and Humanitarian Demining 

A portable handheld mine detection approach to sensor movement is slow and hazardous 
for the individual deminers. Armoured vehicles may not thoroughly protect the occupants 
and may be of only limited usefulness in off -road operations. Most people in the mine 
clearance community would be delighted if the work could be done remotely through 
teleoperated systems or, even better, autonomously through the use of service robots. 
Remote control of most equipment is quite feasible. However, the benefit of mounting a 
mine detector on a remotely controlled vehicle should have careful considerations that lead 
to decide whether the anticipated reduction in risk to the operator justifies the added cost 
and possible reduction in efficiency. A cost analysis should be made to determine to what 
extent remote control approach is a valid solution. 

To increase mine clearance daily performance by improving productivity and accuracy, and 
to increase safety of demining operations and personnel, there is a need for an efficient, 
reliable and cost effective humanitarian mine action equipment with flexible and adaptable 
mobility, and some level of decision making capabilities. Such equipment should have 
selectable sets of mine detectors and work to locate and mark individual mines precisely, and 
at a later stage to neutralize the detected mines. Robotics solutions properly sized with 
suitable modularized mechanized structure and well adapted to local conditions of 
minefields can greatly improve the safety of personnel as well as work efficiency, 
productivity and flexibility. Robotics solution can range from modular components that can 
convert any mine clearing vehicle to a remote-controlled device, to prodding tools connected 
to a robotic arm, and to mobile vehicles with arrays of detection sensors and area mine- 
clearance devices. The targeted robot should have the capability to operate in multi modes. It 
should be possible for someone with only basic training to operate the system. Robots can 
speedup the clearance process when used in combination with handheld mine detection 
tools, and they are going to be useful for quick verification and quality control. To facilitate a 
good robot performance in the demining process, there is a need to employ mechanized 
systems that are able to remove obstructions that deter manual and canine search methods 
without severely disturbing soil. Solving this problem presents challenges in the robotics 
research field and all relevant research areas. Robotics research requires the successful 
integration of a number of disparate technologies that need to have a focus to develop: 

a) Flexible mechanics and modular structures, 

b) Mobility and behavior based control architecture, 

c) Human support functionalities and interaction, 

d) Homogeneous and heterogeneous sensors integration and data fusion, 

e) Different aspect of fast autonomous or semi-autonomous navigation in a dynamic and 
unstructured environment, 

f) Planning, coordination, and cooperation among multi robots, 

g) Wireless connectivity and natural communication with humans, 
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h) Virtual reality and real time interaction to support the planning and logistics of robot 

service, and 
i) Machine intelligence, computation intelligence and advanced signal processing 

algorithms and techniques. 
Furthermore, the use of many robots working and coordinating their movement will 
improve the productivity of overall mine detection and demining process through the use of 
team of robots cooperating and coordinating their work in parallel to enable parallel tasks 
(Gage, 1995; Habib, 1998). 

The possible introduction of robots into demining process can be done through surface 
preparation and marking, speeding-up detection, and mine removal or neutralization. In 
addition, service robots can be used for minefield mapping too. However, the cost of 
applying service robot's technologies and techniques must be justified by the benefits it 
provides. There is no doubt that one of the major benefits would be the safety, by removing 
the operator from the hazardous area. 

It is clear that the development of a unique and universal robot that can operate under 
wide and different terrain and environmental conditions to meet demining requirements 
is not a simple task. In the short term, it appears that the best use of robotics will be as 
mobile platforms with arrays of mine detection sensors and area mine clearance devices. 
Teleoperations are promising but are limited too, because their remote human controllers 
have limited feedback and are unable to drive them effectively in real time. There are 
still some doubts whether such equipment will operate as effectively when the operator 
is at a long distance, or has been removed altogether. Strangely enough, this is 
particularly true for urban areas normally full of rubble, while agricultural areas seem to 
be better, but that is not always true. A possible idea in using robots for demining is to 
design a series of simple and modularized robots, each one capable of performing one of 
the elementary operations that are required to effectively clear a minefield. An 
appropriate mix of such machines should be chosen for each demining task, keeping in 
mind that it is very unlikely that the whole process can be made fully autonomous. It is 
absolutely clear that in many cases, the environment to be dealt with is so hostile that no 
autonomous robot has any chance to be used in mid and short terms. The effort devoted 
to robotic solutions would be more helpful if it is directed at simple equipment 
improvements and low-cost robotic devices to provide some useful improvements in 
safety and cost-effectiveness in the short to medium term. 

Several practical difficulties in using robots for mine clearance have been highlighed 
(Treveylan, 1997). There is little value in a system that makes life safer for the operator but 
which will be less effective at clearing the ground. Accordingly, a serious evaluation and 
analysis should be done along with having efficient design and techniques. The high cost 
and sophisticated technology used in robots which required highly trained personal to 
operate and maintain them are additional factors limiting the possibilities of using robots for 
humanitarian demining. In spite of this, many efforts have been recognized to develop 
effective robots for the purpose to offer cheap and fast solution (Nicoud & Machler, 1996; 
Habib, 2001b). 

Before applying robotics technology for the mine clearance process, it is necessary to specify 
the basic requirements for a robot to have in order to achieve a better performance. These 
requirements include mechanisms, algorithms, functions and use. 
a) It is essential to design a robot that will not easily detonate any mines it might cross on 

its way, i.e, to apply ground pressure that will not exceeds the threshold that sets off the 
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mines in question. Ground pressure is recognized as an important constraint on a 
demining vehicle, because ground pressure is what disturbs the ground and triggers 
many landmines. If a demining vehicle is to safely traverse a minefield, it must exert as 
low ground pressure as possible. Preferably this would be lower than the minimum 
pressure value which would detonate a mine. 

b) The robot should be able to cross safely over the various ground conditions. This can be 
achieved by having adaptable and modular locomotion mechanism both for the mobility 
and structure. The mechanical structure of the robot should be simple, flexible and 
highly reliable. 

c) The robot must be practical, low purchased cost and cheap to run, small, lightweight, 
and portable. 

d) The robot should have efficient surface locomotion concept that is well adapted to 
unstructured environment. The design should assure proper balance between 
maneuverability, stability, speed, and the ability to overcome obstacles. 

e) It should employ multi sensors system for detecting and recognizing different mines. 

f) It should have suitable mechanism for self -recovery for some levels of the problems that 
it might face during navigation and searching for mines. 

g) Design considerations should be given to have a robot that can resist water, sand, 
temperature and humidity. 

h) The mechanical design of the robot should consider practical technology and should be 
as simple and low in technology so that anyone can find and replace and possibly make 
it using locally available materials, such as, bicycle components, bamboo, etc. 
i) The robot should work in more than one operational mode such as teleoperated, semi- 
autonomous, and autonomous modes while keeping the deminer out of physical 
contacts with mine areas. Operator safety should be guaranteed, 
j) It should be capable of withstanding explosive blast without suffering major damage. At 
the minimum the high tech parts of the robot that cannot be replaced locally should be 
well protected, 
k) The robot should be easy to maintain in terms of service and repair by indigenous users. 
Ease of maintenance is built in at the design stage so that if repair is ever necessary it 
may be carried out locally without the use of special test equipment or specialized staff. 
The robots need to be tested and deployed without minimum cost. 
1) Sustaining a reasonable power supply to enable the robot to operate for long period, 
m) Efficient navigation techniques, with sensor based localization in the minefield, and 
man-machine-interfaces including the ergonomy of lightweight portable control stations 
with friendly user interface. 
Research into individual, mine-seeking robots is in the early stages. In their current status, 
they are not an appropriate solution for mine clearance. This is because, their use is 
bounded by sensing devices and techniques improvements, the difficulties facing 
automated solutions raised by the variety of mines and minefields, and the variety of 
terrains in which mine can be found. Examples of such terrains include, dessert, sides of 
mountains, rocky, forest, rice paddy, river banks, plantations, residential areas, etc. Also, 
robotized solutions are yet too expensive to be used for humanitarian demining operations 
in countries like Angola, Afghanistan, Cambodia, etc. 

Many efforts have been recognized to develop an effective robots for the purpose to offer 
cheap and fast solutions. Three main directions can be recognized: Teleoperated machines, 
Multifunctional teleopeated robot, and Demining service robots 
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10. Robotization of Humanitarian Demining 

This section highlights some of the main efforts that aim to robotize the process of 
humanitarian demining: 

10.1 Teleoperated Machines 

10.1.1 Light-Flail 

Smaller and cheaper versions of the flail systems are developed with chains attached to a 
spinning rotor to beat the ground and integrated with remotely controlled, line-of-sight, 
skid loader chases. The use of light-flails aim to safely clear light to medium vegetation, 
neutralize AP-mines and UXOs from footpaths and off-road areas, and assist in area 
reduction of minefield (See Fig. 1). These machines are developed to provide a capability to 
remotely clear AP mines and proof areas that have been cleared (Humanitarian Demining 
Developmental Technologies, 1998; Geneva Centre for Humanitarian Demining, 2006). The 
design of such machines was in particular for dealing with vegetation clearance and 
tripwires as a precursor to accelerate manual clearance. These flail systems are not designed 
for heavily vegetated or extremely rough terrain. Some systems can clear AP mines from off- 
road locations and areas that are not accessible by larger mechanical mine clearing 
equipment. The light-Flail can defeat bounding, tripwire, fuzzed, and simple pressure AP 
mines. In addition, these machines have flail clearance depth between 150mm and 200mm 
and range of working width between 1.4m and 2.22m. These machines are designed to 
withstand blasts up to 9 kg of TNT. They are remotely controlled up to a range of 5,000m 
through feedback sensors and up to 500m away (line-of-sight distance) if it is working in an 
open space. An armored hood is available to protect these machines against AP mine blasts. 
Furthermore, there are set of tracks for installation over the tires when working in soft soil 
conditions to improve traction. 

Different machines made by different manufacturers with almost similar concept are available and 
have been used in real minefields. Some of these are (Humanitarian Demining Developmental 
Technologies, 1998; Geneva Centre for Humanitarian Demining, 2006; Croatia Mine Action Centre, 
2002; Danielsson et al., 2003; Danielsson et al., 2004; Leach, 2004): 

a) Two machines of Armtrac 25 are in service with the UK Ministry of Defense with no 
information for actual usage in a real minefield, 

b) More than 110 Bozena machines have been produced. These machines have been, or are 
currently, in service in Afghanistan, Albania, Angola, Azerbaijan, Bosnia and 
Herzegovina, Cambodia, Czech Republic, Eritrea, Ethiopia, Iraq, Kenya, Kosovo, 
Lebanon, The Netherlands, Poland, Slovakia, Sri Lanka, and Thailand, 

c) The Compact Minecat 140 was developed in 2001 as a direct follow-up improvement of 
the MineCat 230 and has not yet been used in real minefields, 

d) There are 62 MV-4 light flails have been purchased by various organizations/ demining 
companies. Some of the organizations are, US Army (21 units), Swedish Army (5 units), 
Croatian Army (2 units), Irish Army (2 units), International Mine Action Training Centre 
(IMATC) Kenya (1 unit), Croatian Mine Action Centre (CROMAC) (4 units), Iraqi 
National Mine Action Authority (4 units), Norwegian People's Aid (NPA) (3 units), 
Swiss Foundation for Mine Action (FSD) (5 units), etc, 

e) Mini-Flails have been tested extensively in Kuwait, Bosnia, Kosovo, and Jordan. Currently, Six 
Mini-Flails are deployed today in the Balkans, and four systems are deployed in Afghanistan. 
The new version v Mini-Deminer v incorporates improvements to the problems associated with 
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the U.S. Army's original Mini-Flail identified during field evaluations. Development testing of 
the Mini-Deminer took place during the spring and summer of 1999, and 
f) There is no information available by the manufacturer on the actual usage of Diana 44T 

machine in real minefields. 
All light flail machines are featured by, small and compact in size, ease to transport on a 
light trailer, remotely controlled, ease of maintenance and repair, powerful engine with 
efficient cooling system, etc. 

Light flail machines have difficulties to operate with precision from a long distance (this applies 
to all remotely controlled machines), as they require line of sight operation with suitabel 
feedback. The ground flailing systems creates large dust clouds and the high vegetation will 
restrict operator's view on the machine. They also exhibit difficulty in flailing in soft soil, and can 
inadvertently scatter mines into previously cleared areas. All machines are not intended to be 
used in areas where AT mines are present, and they may not be usable in steep or rocky terrain. 




(a) Armtrac 25. 
(Armtrac Ltd 
Kingdom) 



(b) Bozena 4 
United (WAY Industry 
Slovak Republic) 



(c) Mini-flail. 
J.S. Co, (US Department of 
Defense) 




(d) Diana 44T 
(Hontstav S.R.O., Slovak 
Republic) 



(e) Minecat 140. 
(Norwegian Demining 
Consortium) 



(f) The MV-4. 
(DOK-ING d.o.o., Croatia) 



Fig.l. Different types of light flails in action. 



10.1.2 Remotely Operated Vehicles (Kentree Limited) 

Kentree Limited has been designing and manufacturing variety of remotely operated vehicles. 
Hobo was the early developed vehicle and it has a reasonable maneuverability, 6 robust wheels to 
allow carriage goes over obstacles and through water. Many updates have been introduced to 
meet the continued requirements in Explosive Ordnance Disposal (EOD)/ Improvised Explosive 
Device Disposal (IEDD) applications and those required in battle zones, nuclear, chemical or fire 
fighting situations. The most apparent are the articulating rear axle and the Radio Control. The 
tracked chassis has a front ramp section which lowers to provide a variable footprint. With this 
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additional traction, the vehicle negotiates slopes, stairs and steps with ease. Hobot is the track 
version of Hobo for use in areas where tracks are the required option as in certain nuclear or 
chemical environments. The dimension of Hobo L3A15 is L= 148.3cm, W= 70.76cm and H= 
88.81cm, the vehicle weight when empty is 228 kg, the payload of the arm is 30 kg, and the 
maximum speed is 4km/ h. Other teleoperated vehicle developed by Kentree includes, Vegabond, 
Rambler, Max, Brat, Tramp and Imp. 

One of the latest additions to the Kentree family of vehicles is the "Thrasher" mobile vehicle 
designed for the purpose of demining. Kentree and the Irish armed forces are developing 
Thrasher as cost-effective solution for demining operations. Thrasher is small and it is capable of 
dealing with narrow laneways. The remotely controlled route clearance flail system is aimed at 
clearing a 4 feet wide path of booby traps and AP mines to allow safe personnel passage. The 
vehicle can also be fitted with an offset rear flail attachment, to increase the beat area to 8 feet. This 
will allow the access of small transport vehicles. The ROV can be controlled via secure radio link 
from the front passenger seat of a jeep by means of a laptop control console with video feed to 
virtual reality goggles. Alternatively, it may be operated by backpack style system with hand 
control for foot-mounted demining operations. No information for demining testing and 
evaluation is available. Figure 2 shows Hobo, Hobot and Thrasher robots. 




(a) The wheeled Hobo. 



(b) The wheeled Hobo. (c) the tracked Hobot. 




(d) The Brat. (e) Thrasher. 

Fig. 2. Remotely operated vehicles from Kentree. 



10.2 Multi Functional Teleoperated Robots 

10.2.1 Demining mobile robot MR-2 (Engineering Service Incorporation (ESI)) 

MR-2 is an off -road, modular, teleoperated, multi-sensor mobile platform designed to detect 
landmines, including those with minimal metal content, and UXO. MR-2 is a modular 
system comprising a remotely operated vehicle (ROV), control unit, MR-1 robotic arm for 
scanning, laser range camera and metal detector (ESI, 2003). MR-2 uses only one metal 
detector (of-the-shelf unit that can be easily detached and used manually), and combines the 
latest laser/ ultrasonic based terrain imaging technology that allows the metal detector to 
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adaptively follows the terrain surface while avoiding obstacles. MR-2 can perform 
neutralization of landmines using MR-1 arm under the supervision of remotely located 
operator. MR-1 is a ragged modular dexterous robotic arm (See Fig. 3). The ROV is capable 
of turning 360 degrees in 1.5 m wide hallway, traversing virtually any terrain up to 45 
degrees in slope, over 70 cm ditches, curbs, etc. It operates either with wheel or track and 
quick mount/ dismount tracks over wheels. MR-2 works at high-speed scanning (up to 5 
km/ hour) with wide detection path (about 3 m). The MR-2 is an autonomous mine 
detection system that operates at high speed with minimum logistic burden. The MR-2 is a 
high cost and heavy robot that is designed to search for mines in terrain with rich vegetation 
stones, sand, puddles and various obstacles. The open architecture of MR-2 allows 
expansion with generic and custom-made modules (semi-autonomous navigation, pre- 
programmed motion, landmine detection, etc.). Sensor payloads can be extended to include 
a metal detection array, an infrared imager, GPR and a thermal neutron activation detector. 
Data fusion methodologies are used to combine the discrete detector outputs for 
presentation to the operator. No evaluation and testing results in relation to demining are 
available. 




(a) The MR-Robot. (b) Laser/US imaging, (c) Metal detector, (d) MR-2 with MR-1. 

Fig.3. The MR-2 demining mobile robot. 



10.2.2 Enhanced Tele-Operated Ordnance Disposal System (ETODS) 

(OAO Corporation, Robotics Division) 
The Enhanced Teleoperated Ordnance Disposal System (ETODS) is a remotely controlled 
teleoperated system that is based on a modified commercial skid loader with a modular 
tooling interface which can be field configured to provide the abilities to remotely clear light 
vegetation, detect buried unexploded ordnance (UXO) & landmines, excavate, manipulate, 
and neutralize UXO & landmines mines, to address the need of various mechanical 
clearance activities associated with humanitarian demining (Eisenhauer et al., 1999). ETODS 
has an integrated blast shield and solid tires. 

ETODS includes a heavy vegetation cutter and a rapidly interchangeable arm with 
specialized attachments for landmine excavation. Attachments include an air knife for 
excavation of landmines, a bucket for soil removal, and a gripper arm to manipulate certain 
targets. Remote control capability combined with a differential GPS subsystem and on- 
board cameras enable the system to navigate within a minefield to locations of previously 
marked mines. Mines or suspicious objects already marked or identified with GPS 
coordinates can be checked and confirmed with an on-board commercial detector and then 
excavated with a modified commercial backhoe, an air knife, excavation bucket, or gripper 
attachment. ETODS was developed and configured for the US DoD humanitarian demining 
research and development Program starting in 1995. It has been through many field test 
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activities, and they found it suitable for use in humanitarian demining (HD) operations. The 
HD issues that have been evaluated include accuracy, repeatability, and feasibility of usage 
in remote environments. In relation to vegetation cutting, three attachments have been 
tested. One front mounted bush hog and two side mounted boom mowers. In this case, the 
HD issues that have been evaluated include the ability to cut dense undergrowth, the proper 
preparation of the ground for ensuing detection activities, and the ability of the operator to 
effectively and efficiently clear an area under remote control. As for commercial backhoe 
that can be field mounted to the ETODS, the HD issues that have been evaluated include the 
effectiveness and efficiency of locating and excavating mines, operator training 
requirements, inadvertent detonation rates, techniques for deeper excavations, techniques to 
identify mines and their status (e.g. booby trapped), and blast survivability/ repair. A chain 
flail attachment converts the ETODS into a system capable of clearing AP mines through 
detonation, and for this case the HD issues that have been evaluated include the minimum 
sized mine cleared, depth of clearance, effectiveness of clearance, speed of clearance, and 
blast survivability/ repair. During testing, ETODS was subjected to a 12 lb. TNT blast 
replicating an AT mine detonation. ETODS drove away with field repairable damage. 
ETODS has proven effective in detonating M14 AP mines and is survivable through 
repeated 1.0 lb. TNT detonations (OAO-Robotics, website). TODS provides safe, effective 
delivery of tools necessary for the clearance of landmines and UXO. ETODS is simple, 
rugged, and can provide a high technology indigenous demining capability in remote 
environments. 

The ETODS has completed operational field evaluations in Jordan and Egypt, where it was found 
to have several significant limitations that make it less than suitable for humanitarian demining 
operations (Figure 4 shows the ETODS is action). These include the tendency to become mired in 
mud or desert sand conditions, as well as the requirement for significant training to develop tele- 
operation skills (Department of Defense, Development Technologies, 2001-2002). 




Fig. 4. The ETODS in action. 



10.2.3 TEMPEST. (Development Technology Workshop (DTW) 

TEMPEST is designed to safely clear light to medium vegetation, clear tripwire fuzed mines, 
and assist in area reduction as a precursor to accelerated manual clearance. DTW began 
production of the TEMPEST Mk I in 1998-99 in which it was designed purely as a 
vegetation-cutting device, and currently, the TEMPEST Mk V is in production. The 
TEMPEST Mk V is a remotely controlled, lightweight multi-tool system with vegetation 
cutting and trip wire clearing abilities (See Fig.5). 

TEMPEST is a low cost, small size and light weight radio controlled AP mine blast-protected 
multi purpose ground based system. These features aim to ease of transport and agility over 
difficult terrain. It can support a variety of interchangeable clearance heads to clear vegetation, 
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removal of metal fragmentations by using large and small magnets for the removal of metal 
fragmentations, engage the ground with flail head, and neutralize tripwires, etc. It is designed to 
clear AP mines from off-road areas inaccessible to large-area mine clearers. The TEMPEST 
system consists of a diesel powered hydraulically driven chassis, a radio control subsystem, and 
each of its four hydrostatic wheels is driven by an independent motor to improve 
maneuverability. The wheels are easy to remove, repair and replace. The TEMPEST also has a 
1.2-meter wide horizontal chain flail with vegetation cutting tips, and an adaptable flail head 
with hydraulic feedback system that can sense the load on the flail, i.e., the operator can set the 
speed control to maximum and the TEMPEST will automatically control its cutting rate and 
drive speed, and progress accordingly. The TEMPEST s ground engagement flail is designed to 
dig into the soil in order to destroy or expose mines by cutting 10 cm deep into the ground to 
initiate surface and sub surface mines at that level. Its V-shaped chassis and sacrificial wheels 
minimize damage from anti-personnel mine or UXO detonation and provide some protection 
against anti-tank mines. TEMPEST'S vertical axis "slasher" is capable of cutting through difficult 
vegetation such as bamboo and vines and its large magnetic array is capable of extracting ferrous 
material from the ground. It is able to clear up to 200m 2 / h of light vegetation (500mm tall thick 
grass) and to cut 100 mm tree in 3-4 minutes. TEMPEST is featured by ease of operation, 
maintenance, and repair. 

TEMPEST is inexpensive to purchase and operate relative to other vegetation clearance 
systems. Currently, the TEMPEST is produced in Cambodia as well as the United Kingdom, 
thus representing a regional capability in Southeast Asia (Department of Defense, 
Development Technologies, 2001-2002). 

The TEMPEST is an excellent example of how an operational evaluation can lead to 
improvements that realize the potential of a prototype design. The early prototype of 
TEMPEST underwent extensive tests in Cambodia for AP and AT mines. The 
TEMPEST began an operational evaluation in Thailand in January 2001. Although it 
was effective at clearing vegetation in mined areas, Thai operators identified 
overheating problems. The unit's promising performance warranted the investment of 
funds to improve the system. TEMPEST Mk IV has been tested in Mozambique during 
2003. The actual use of TEMEST systems and the continuous evaluation results in 
having TEMPEST Mk V as a reliable system with more speed and engine power 
capacity compare to the previous versions. As evaluated by the manufacturer, the 
hydraulic hoses are vulnerable to fragmentation attacks, and the machine is not intended to 
be used in areas where AT mines are present. As evaluated by deminers, the TEMPEST 
requires the operator to maintain direct line of sight with the system from a minimum of 50 
meters and the operator can only be this close if behind the system's portable shield. This 
poses a problem in dense vegetation or rolling terrain. The TEMPEST has limited traction on 
wet muddy terrain due to the steel wheels clogging with mud. The machine has the ability 
to clear both mines and vegetation, even though with limitations. The ground flailing 
system creates large dust clouds. The view of the operator on the machine can be restricted 
and the air filters can be clogged (Leach et al., 2005). 

Currently, there are now 25 machines operating in Angola, Bosnia, Cambodia, DR Congo, 
Mozambique, Sri Lanka and Thailand. The TEMPEST is currently used by seven demining 
organizations around the world (Geneva Centre for Humanitarian Demining, 2006). The 
new TEMPEST Mk VI will mitigate the highlighted problems by use of a new remote control 
system and the integration of tracks in place of the steel wheels to enable the vehicle to 
operate on most soil conditions and terrains. 
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Fig. 5. Tempest during operational field evaluation. 



10.3 Demining Service Robots 

10.3.1 Three wheels Dervish robot (University of Edinburgh/ UK) 

Dervish was originally designed to bypass the problem of mine detection by deliberately 
rolling over the mines with mine-resistant wheels. The Dervish is a remotely-controlled 
wheeled vehicle designed to detect and detonate AP mines with charge weights up to 250 
grams that is equivalent to the largest size of AP mines. It is a three- wheeled vehicle with 
wheel axles pointing to the center of a triangle. The weight of Dervish closely emulates (a 
little more than) the ground loading of a human leg (Salter & Gibson, 1999). But, becasue 
of its low weight, Dervish will not explode AT mines. The wheels are placed at 120 
degrees from each other. The Dervish drive uses three variable-displacement computer- 
controlled hydraulic pumps driven by a 340 cc Honda engine, and controlled by a micro- 
processor to drive a Danfoss hydraulic motor at each wheel. The steel wheels weight 
about 80 kg and are 4-6 cm thick. Due to the position of the wheels, if all Dervish wheels 
were driven at the same speed then it would merely rotate about its center and make no 
forward progress. However, carefully timed, small, cyclical variations of wheel speed 
make the Dervish wheels describe spirals and progressively translate in a chosen direction 
so that every point in its path is covered, twice, by a loading of about 90 kg in a pattern of 
overlapping circles. Repeatedly locking one wheel and driving the other two wheels spins 
the machine through 120 degrees about the locked one and allows traversing. Dervish has 
a very open steel frame with all members oblique to the path of blast fragments. It 
effectively has a zero-radius turning circle. A wide path can hence be stamped by radio- 
control. Figure 6 shows Dervish and illustrates the spiral movements of the robot. It is 
claimed by the designer that in case of mine explosion, the wheel and the compact 
hydraulic motor should resist. The tetrahedral structure linking the three wheels and the 
central power source will be easily repaired. 

In normal mine-detonating mode, the Dervish advances at about one meter a minute, a rate 
set by the requirement that there should be no mine-sized gaps between its wheel tracks, i.e, 
covering the ground at intervals of only 3cm to avoid any mine-sized gaps between its 
wheel tracks. A possible change to the wheel design may increase this by a factor of three. 
With its design structure, it can sweep a 5 meter wide track with a possible coverage of 300- 
900 square meters per hour. The machine is designed for the clearance of agricultural land. 
It can operate on open, uneven, or moderately sloping ground. All the electronic equipment 
is fitted into steel tubes made from old nitrogen bottles with carefully-machined O-ring seals 
and uses military specification connectors. The Dervish can carry a metal detector placed in 
a thorn-resistant protective shroud with the sensor head just inboard of the wheel radius at 
60 degrees from a wheel. Other sensors for non-metallic targets especially ones that respond 
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to explosives in gram quantities have not been introduced. In a test with a 10kg charge, 
damage was confined to one corner while the axle and bearings from that test are still in use. 
The repair cost would be a few hundred dollars. The main limitations of this robot are: not 
suitable for difficult terrain, hard to navigate, blast-resistant wheels are unsuited to very soft 
ground, and the inability of the robot with its particular wheel configuration and available 
power to have enough torque to get out of a hole after a mine blast. This has prompted the 
team to work on a future complementary design aimed purely at sensor movement with no 
mine detonation. 




Dervish Mk III (April 93) 



Fig. 6. The DERVISH robot. 



10.3.2 PEMEX-BE (PErsonal Mine EXplorer) (LAMI-EPFL/Switzerland) 

Pemex is a low cost solution for carrying a mine sensor and exploring automatically an area. 
Pemex is a two-wheeled robot built uses mountain bicycle wheels and aims to investigate 
cross-country navigation and to evaluate sensors for the detection of AP mines (See Fig. 7). 
It is a lightweight vehicle (less than 16 kg) and exerts a maximum force of 6 kg on the 
ground that is not supposed to trigger any of AP mines it detects. The wheels are driven by 
90 W DC motors from Maxon with 1:72 reductors aiming to give to the robot a maximum 
speed of 6 km/h power it. When searching for mines the Pemex head oscillates right and 
left in a zigzag movement covering a 1-meter wide path (Nicoud & Habib, 1995; Nicoud, 
1996). The on-board 68331 microprocessor permits autonomous or teleoperated navigation. 
Polaroid and Sharp PSD ultrasonic sonar sensors detect obstacles. The mine sensor head 
currently contains as a metal detector. It is intended to be integrated a combination of a 
metal detector (MD) and a ground-penetrating radar (GPR) that have been evaluated in real 
minefield. The ERA radar was selected in early 1996, and different metal detectors brands 
from (Schiebel-Austria, Foerster-Germany and Ebinger-Germany) were used and tested 
(Nicoud at el., 1998). Pemex has rechargeable batteries that can provide 60 minutes of 
autonomy. 

Mined terrain is often overgrown with dense vegetation. Pemex-BE's mountain bike wheels 
allow it to move in high grass. With climbing cleats mounted on its wheels, Pemex-BE can 
climb irregular slopes of 20° to 30°. It can also climb stairs. The wheels go first when 
climbing to prevent the sensor package leaving the ground. Pemex is equipped with 
optional water wings that enable it to float and swim. This allows it to operate in 
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environments such as rice paddies and, on land, reduces the pressure on the ground when 
searching for very sensitive pressure-triggered mines. For transport, the wheels can be 
removed and attached to the sides of the main chassis. All components can be packed and 
easily carried by one person. 




Fig. 7. PEMEX - BE (PErsonal Mine EXplorer). 



10.3.3 Shrimp Robot (EPFL/Switzerland) 

As part of the field and space robotics activities at the Autonomous Systems Lab (ASL) of 
EPFL-Switzerland, an innovative robot structure has been developed. The first prototype 
is called the " Shrimp Robot". Shrimp is a high mobility 6-wheels mobile platform. One 
wheel is front-mounted on an articulated fork, one wheel in rear directly connected to the 
body and two wheels are mounted on each of two lateral bogies. The total weight of this 
first prototype is 3.1 kg including 600 g of batteries and a 1.75 W DC motor powers each 
wheel. The dimensions are L 60 cm x W 35 cm x H 23 cm; the ground clearance is 15 cm. 
Shrimp as a new mobile platform shows excellent off-road abilities overcoming rocks 
even with a single bogie. Shrimp adapts its structure purely passively during motion to 
insure its stability. This allows very simple control strategy as well as low power 
consumption. The secret of its high mobility lies in the parallel architecture of the front 
fork and of the bogies ((Estier et al., 2000a; Estier et al., 2000b). With its passive structure, 
Shrimp does not need to actively sense obstacles for climbing them. Instead, it simply 
moves forward and lets its mechanical structure adapt to the terrain profile. With a frontal 
inclination of 40 degrees, Shrimp is able to passively overcome steps of twice its wheel 
diameter, to climb stairs or to move in very rough terrain. Shrimp has not been used yet in 
demining operation, but it can be considered an attractive candidate because of its well- 
adapted locomotion concept and the excellent climbing and steering capabilities that 
allow high ground clearance while it has very good stability on different types of rough 
terrain. In May 2001, the developer announced version 3 of the robot, Shrimp III (See Fig. 
8). This version is powered by 6 motors integrated inside the wheels and steered by two 
servos. This robot is able to turn on the spot. It is built in anodized Aluminium and it is 
equipped with modular electronics. 




Fig. 8. The Shrimp III Robot. 
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10.3.4 Automatic Mechanical Means of Manual Land Mine Detection 

The aim is to design an automated, single or multiple-prodding device that can be mounted 
installed in front of a remotely controlled all terrain vehicles. In this regards, at the 
suggestion of The Defence Research Establishment Suffield (DRES), the 1996 senior design 
project the University of Alberta was to design innovative mechanical method to detect non- 
metallic landmines (Fyfe, 1996). The developed design tries to emulate and multiply the 
performance of manual prodding done by human operator. The design consists of an 
automated and hydraulically actuated multiple-prodding device designed to be mounted 
either in front of a BISON armoured personnel or in front of a remotely controlled all terrain 
vehicle called ARGO. The detection unit consists of a frame, traversing rack and multiple 
probes. Each of the 41 or 8 probes (depending on the design) used to penetrate the ground, 
is individually mounted on a hydraulic cylinder (See Fig.9.). The hydraulic fluid pressure in 
each cylinder is continuously monitored by a computer data acquisition system. When the 
probe strikes the soil or a solid object, the pressure in the cylinder rises in proportion to the 
force on the probe. Once this pressure rises above a threshold value, a solid object is 
determined to be present. A solenoid valve controlled by the computer releases the pressure 
in the cylinder, thus stopping the probe from further motion. This valve is quick enough to 
stop the cylinder in order to prevent the accidental detonation of the suspected mine. Based 
on the probe separation distance, this system ensures that no landmine is going to be missed 
by passing between the probes. 




Fig. 9. The design of multiple mechanical means of manual prodding. 

A similar approach has been developed (Dawson-Howe & Williams, 1997). They have 
assembled a lab prototype, as shown in Fig. 10, intended to demonstrate the feasibility of 
automatic probing using on an XY table for the motion (to be fixed on a mobile platform at a 
later stage), together with a linear actuator, a force sensor and a sharpened steel rod. Probing 
test was done on an area of 50cm x 50cm and the probing was done at an angle of 30 degrees. 
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Fig. 10. A laboratory prototype of a single mechanical means of manual prodding. 
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10.3.5 AMRU and Tridem (I and II) (Belgium HUDEM) 

The Belgian joint research program for Humanitarian DEMining (HUDEM) aims to enhance 
mine detection by a multi-sensor approach, speed up the minefield perimeter determination and 
map the minefields by robotic platform. Several mobile scanning systems have been developed, 
such as the AMRU (Autonomy of Mobile Robots in Unstructured environments) series 1-4, have 
been modified from previously developed walking mobile robots by Belgium Royal Military. 
One of the main purposes of developing such robots was to achieve low-cost machines. In 
order to meet this constraint, simple mechanical systems for the legs were used and high 
cost servomotors were replaced by pneumatic and other actuation systems. A simple but 
robust digital control was implemented using industrial PLCs for the early versions. 
AMRU-1 is a sliding robot actuated by rodless pneumatically cylinders with the capacity 
to have 4*90 degree indexed rotation. When the metal detector detects something, the 
robot stops and an alarm is reported to the operator. The robot is equipped with a 
detection scanner. This robot has poor adaptability to irregular terrain with limited 
flexibility. AMRU 2 is a six-legged electro-pneumatic robot. Each leg has 3 degrees of 
freedom rotating around a horizontal axis allowing the transport/ transfer phase, a 
rotation around a horizontal axis used for the radial elongation of the legs and a linear 
translation allowing the choice of the height of the foot. The first two dofs are obtained by 
use of rotating double acting pneumatic motors plus double acting cylinders. Other 
versions have been developed (AMRU 3 and 4) but they are still waiting for testing. The 
next generation AMRU 5 has 6 legs. 




(d) Tridem I (f ) Tridem II (g) Tridem II with Metal Detector 

Fig. 11 Different versions of AMRU and Tridem robots. 

In order to obtain a better mobility, the Tridem robot series have been developed. This 
series of robots has been equipped with three independent modular drive/ steer wheels. 
Each wheel has 2 electrical motors. A triangular frame connects the wheels. This frame 
supports holding the control electronics and the batteries. The robot has been design to 
have a 20-kg payload and a speed of 0.1 m/sec. Two versions of this robot have been 
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developed (Tridem I and II). Figure 11 illustrates different versions of AMRU and Tridem 
robots. 



10.3.6 WHEELEG (University of CATANIA, Italy) 

Since 1998, the WHEELEG robot has been designed and built for the purpose to investigate 
the capabilities of a hybrid wheeled and legged locomotion structure in rough terrain 
(Muscato & Nunnari, 1999; Guccione & Muscato, 2003). The main idea underlying the 
wheeled-legged robot is the use of rear wheels to carry most of the weight and front legs to 
improve surface grip on climbing surface and overcome obstacles (See Fig. 12). This robot 
has two pneumatically actuated front legs with sliding motion, each one with three degrees 
of freedom, and two rear wheels independently actuated by using two distinct DC motors. 
The robot dimensions are Width=66cm, Length=lllcm, and Height=40cm. The WHEELEG 
has six ST52E301 Fuzzy microcontrollers for the control of the pistons, two DSP HCTL1100 
for the control of the wheels and a PENTIUM 200MHz microprocessor for the global 
trajectory control and the communications with the user. Preliminary navigation tests have 
been performed showing that WHEELEG cannot only walk but also run. During walking, 
the robot can overcome obstacles up to 20 cm high, and it can climb over irregular terrain. 
Possible applications that have been envisaged are humanitarian demining, exploration of 
unstructured environments like volcanoes etc. The robot mobility and maneuverability is 
limited, no demining sensors have been used, and no demining testing and evaluation has 
been reported. 




(a) WHEELEG prototype 
Fig. 12. The WEELEG Robot. 



(b) WHEELEG tested on Etna volcano 



10.3.7 Spiral Terrain Autonomous Robot (STAR) 

(Lawrence Livermore National Laboratory (LLNL) 

An autonomous vehicle has been developed for versatile use in hostile environments to help 
reduce the risk to personnel and equipment during high-risk missions. In 1996 LLNL was in 
the process of developing the Spiral Track Autonomous Robot (STAR), as an electro- 
mechanical vehicle that can be fitted with multiple sensor packages to complete a variety of 
desired missions. STAR is a versatile and manoeuvrable multi-terrain mobile robot that can 
to be used as an intelligent search and rescue vehicle to negotiate fragile and hostile 
environments (Perez, 1996). STAR can help with search and rescue missions after disasters, 
or explore the surfaces of other planets (See Fig. 13). 
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Although four-wheel and track vehicles work well, they are limited in negotiating 
saturated terrain, steep hills and soft soils. The two key mechanical components in the 
structure of STAR are the frame assembly and the two Archimedes screws. The 
mechanical frame is made of hollow aluminum cylinders welded together with an 
aluminum faceplate on each end. The second key mechanical component of the STAR 
is the screw drive The STAR rolls on a pair of giant Archimedes screws (one left-hand 
and one right-hand) that serve as the drive mechanism in contact with the local 
environment to propel itself along the ground. The screws take advantage of ground 
forces. Rotating the screws in different rotational combinations causes the system to 
instantly translate and/ or rotate as desired in four possible directions, and to turn 
with a zero turning radius. When they rotate in opposite directions, the robot rumbles 
forward. When they rotate in the same direction, it scuttles sideways, and when one 
screw turns while the other holds still, the screw-bot deftly pirouettes. Versatility in 
directional travel gives the system flexibility to operate in extremely restricted 
quarters not accessible to much larger pieces of equipment. Furthermore, the 
Archimedes screws give the vehicle enough buoyancy to negotiate saturated terrain. 
In water, the hollow screws float and push like propellers. The STAR is compact, 
measuring 38 inches square and 30 inches high; it has a low centre-of-gravity 
allowing the system to climb steep terrains not accessible to other hostile environment 
hardware. 




Fig. 13. The STAR robot in different situations. 

The STAR is also equipped with a complete on-board electronic control system, 
data/ video communication links, and software to provide the STAR with enough 
intelligence and capabilities to operate remotely or autonomously. During remote 
operation, the operator controls the robot from a remote station using wireless data 
link and control system software resident in a laptop computer. The operator is able to 
view the surrounding environment using the wireless video link and camera system. 
Remote operation mode is desirous when personnel must enter an unsecured hostile 
environment that may contain nerve gases, radiation, etc. Ultrasonic sensors are 
mounted around the external perimeter of the robot to provide collision-avoidance 
capabilities during remote and autonomous operations. All power is placed on-board 
the system to allow for tether less missions involving distant travel. The system is 
responsible for high-level decision-making, motion control, autonomous path planning, 
and execution. The cost of the STAR is dependent on the sensor package attached. The 
STAR is equipped with a differential GPS system for autonomous operation and it can 
accommodate the Micro-power Impulse Radar (MIR) for landmine detection 
technology developed by LLNL. A disadvantage of STAR is the high friction between 
the screw wheels and the ground, which keeps the machine to a one-and-a-half-mile- 
per-hour speed limit while moving forward or backward. STAR has been studied in 
specific mine projects. The robot is not suitable for environments that are full of rocks. 
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Experiments have shown the ability of STAR to negotiate successfully, hard and soft 
soils, sand, pavement, mud, and water. No demining testing and evaluation were 
reported. 



10.3.8 COMET I, II and III: Six legged Robot (Chiba University in Japan) 

COMET I and II have six legs and is equipped with several sensors for mine detection 
(Nonami, 1998). COMET III has 2 crawler and 6 legs walking/ running robot with two 
arms in the front. It is driven by hydraulic power. The robot weight 990 kg, its length 
4m, width 2.5m, and height 0.8 m. The COMET is made of composite material for legs 
and manipulators like CFRP to reduce the total weight. Currently, COMET-I can walk 
slowly at speed 20m per hour with precise detection mode using six metal detectors. On 
the other hand, COMET-II can walk at speed 300m per hour with precise detection 
mode using the mixed sensor with metal detector and GPR at the tip of the right 
manipulator. COMET robots are equipped with CCD camera, IR camera and laser 
sensor. Different experiments haven been conducted to detect artificially located mines 
based on the use of infrared sensors that can deal with different terrain (Nonami et al., 
2000). 




COMET I 



COMET II 




COMET III 
Fig. 14. Different versions of the six legged mobile robot COMET. 



476 Mobile Robots, Towards New Applications 

11. Conclusions 

The major technical challenge facing the detection of individual mine, is having the ability to 
discriminate landmines from metal debris, natural clutters and other objects without the 
need for vegetation cutting. Future efforts to improve detection should focus on providing a 
discrimination capability that includes the fusion of information coming from multi 
heterogeneous and homogenous sensors and the incorporation of advanced signal 
processing techniques to support real-time processing and decision making. For the purpose 
of mine clearance, there is an urgent need to have cost-effective and efficient clearance 
techniques to clear landmines in all types of terrains. This should be associated with 
neutralization, in which there is a need to develop safe, reliable, and effective methods to 
eliminate the threat of individual mines without moving them. 

Working in a minefield is not an easy task for a robot. Hostile environmental conditions and 
strict requirements dictated by demining procedures make the development of demining 
robots a challenge. Demining robots offer a challenging opportunity for applying original 
concepts of robotic design and control schemes, and in parallel to this there is urgent need to 
develop new mine detection techniques and approaches for sensor integration, data fusion, 
and information processing. 

Difficulties can be recognized in achieving a robot with specifications that can fulfil the 
stated requirements for humanitarian demining. A lot of demining tasks cannot yet be 
carried out by the available robots because of their poor locomotive mechanism and 
mobility in different type of terrains. This is because there is still lack of well-adopted 
locomotion concepts for both outdoor and off-road locomotion. Hence, there is a need to 
develop modular, light-weight, and low-cost mobile platforms that can deal with different 
terrain. Modularized robotic solutions properly sized and adaptable to local minefield 
conditions is the best way to enable reconfiguration that suite the local needs, greatly 
improve safety of personnel as well as improving efficiency. In order to be able to design 
and build successful robot, it is necessary to carefully study conditions and constraints of 
the demining operations. The technologies to be developed should take into account the 
facts that many of the demining operators will have had minimal formal education and 
that the countries where the equipment are to be used will have poor technological 
infrastructure for servicing and maintenance, spare parts storage, operation and 
deployment/ logistics. 

Research into individual, mine-seeking robots is still in the early stages. In their current 
status, they are not an appropriate solution for mine clearance. Due to the gap between 
scientists developing the robots and the deminers in the field, and because none of the 
developed robots (specifically these presented in section 10.3) yet entered a minefield 
for real and continuous mine detection and removal. Several large research efforts have 
failed so far, to develop an effective mine clearance alternative to the existing manual 
technique. Robots have been tried at great expense, but without success yet. There is 
still a large amount of skepticisms on the role and use of autonomous robots for 
demining purposes. Expert in robotics knows too little about the practical challenge of 
demining: hence the robot is designed like all other autonomous robots attempting to 
navigate an unknown environment. Although some aspects of navigation may be 
extended to demining robots, it will be more reliable if robots were designed 
specifically for the purpose of landmine detection than as an after thought. 
Understanding the current and previous failed research efforts may help to avoid 
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similar mistakes. Detecting and removing AP mines seems to be a perfect challenge for 
robots. But, this requires to have a good understanding of the problem and a careful 
analysis must filter the goals in order to avoid deception and increase the possibility of 
achieving results. 

The approach to solve the humanitarian demining problem and fulfill its needs requires 
a strategy for research and development with both short and long-term components. In 
the short and mid terms, robots can help to accelerate searching and marking mines. In 
addition, it can be helpful to be used for quality assurance stage for verification 
purposes. High cost and high tech features are additional constraints in using robots for 
demining. Any single breakthrough in technology should be viewed as yet another tool 
available for use in the demining process, and it may not be appropriate under all 
conditions. Furthermore, careful study of the limitations of any tool with regard to the 
location and environment is critical; not all high-tech solutions may be workable at all 
places. The knowledge required to operate a machine may not match the skill level of 
the deminers, many of whom are drawn from the local public. In addition, cost of 
maintenance, spare parts and its availability are critical parameters too. While current 
technology may be slightly effective, it is far too limited to fully address the huge mine 
problem facing the world. 
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1. Introduction 

It is said that 60 - 70 million mines are currently laid in over 50 countries throughout the 
world (Shimoi, 2002). These mines not only injure people directly, but also negatively 
influence the lives of inhabitants and restoration of the economy because of their consequent 
restriction of farmland. Such mines are categorized into two types: ATMs (anti-tank mines) 
and APMs (anti-personnel mines). The purpose of ATMs is to destroy tanks and vehicles, 
while that of the APMs is to injure or kill people. The APMs are cheap to produce, while 
APM removal is quite expensive. Also, the APMs maintain their destructive power for a 
long time. To mitigate the current tragic situation, an urgent "humanitarian demining 
operation" (with a demining rate of more than 99.7%) is now required all over the world 
(Shimoi, 2002). 

The demining work mainly depends on hazardous manual removal by humans; it presents 
serious safety and efficiency issues. For increased safety and efficiency, some large-sized 
machines have been developed. For example, the German MgM Rotar rotates a cylindrical 
cage attached in front of the body and separates mines from soil (Geneva International 
Centre for Humanitarian Demining, 2002; Shibata, 2001). The RHINO Earth Tiller, also made 
in Germany, has a large-sized rotor in front of the body; it crushes mines while tilling soil 
(see Fig. 1). The advantages of MgM Rotar and RHINO are a high clearance capability (99%) 
and high efficiency respectively. 

In Japan, Yamanashi Hitachi Construction Machinery Co., Ltd. has developed a demining 
machine based on a hydraulic shovel. A rotary cutter attached to the end of the arm destroys 
mines; the cutter is also used for cutting grasses and bushes. Although many machines with 
various techniques have been developed, a comprehensive solution that is superior to 
human manual removal remains elusive. Salient problems are the demining rate, limitation 
of demining area (MgM Rotar), prohibitive weight and limitation of mine type (RHINO Earth 
Tiller), and demining efficiency (MgM Rotar, and the demining machine made by Yamanashi 
Hitachi Construction Machinery Co., Ltd.). Because those machines are operated manually or 
by remote control, expert operators are required for each machine. Also, working hours are 
limited. 

Recently, various demining robots have been developing mainly at universities. Hirose et 
al. have developed a probe-type mine detecting sensor that replaces a conventional prod 
(Kama et al., 2000). It increases safety and reliability. They have also developed a 
quadruped walking robot, some snake-type robots, mechanical master-slave hands to 
remove landmines, and robotic system with pantograph manipulator (Hirose et al., 2001a; 
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Hirose et al., 2001b; Furihata et al., 2004; Tojo et al., 2004). Nonami et al. have developed a 
locomotion robot with six legs for mine detection (Shiraishi et aL, 2002). A highly sensitive 
metal detector installed on the bottom of each foot detects mines and marks the ground. 
Ushijima et al. proposes a mine detecting system using an airship (Ushijima, 2001). On this 
system, the airship has a control system and a detecting system for mines using 
electromagnetic waves; it flies over the minefield autonomously. These studies mainly 
address mine detection; it is difficult to infer that they effectively consider all processes 
from detection to disposal. 




Fig. 1. Mine removal machine (RHINO Earth Tiller). 

This study proposes an excavation- type demining robot "PEACE" for farmland aiming at 
"complete removal" and "automation"; it also presents the possibility of its realization. The 
robot has a large bucket in front of the body and can travel while maintaining a target depth 
by tilting the bucket. The robot takes soil into the body and crushes the soil, which includes 
mines. It then removes broken mine fragments and restores the soil, previously polluted by 
mines, to a clean condition. In the process, the soil is cultivated, so the land is available for 
farm use immediately. Expert robot operators are not required; the robot works all day long 
because it can be controlled autonomously. 

Section 2 presents the conceptual design of the excavation-type demining robot "PEACE". 
Section 3 describes robot kinematics and trajectory planning. In Section 4, the optimal depth 
of the excavation is discussed. Section 5 shows experimental results of traveling with 
digging soil by a scale model of the robot. In Section 6, the structure of the crusher and 
parameters for crush process are discussed through several experiments. Finally, Section 7 
contains summary and future works. 



2. Conceptual Design of PEACE 

We propose an excavation- type demining robot for farmland aiming at "complete 
removal" and "automation" (Mori et al., 2003, Mori et al., 2005). The reason why we 
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choose farmland as the demining area is as follows: farmland is such an area where local 
people cannot help entering to live, so it should be given the highest priority (Jimbo, 
1997). Needless to say, the first keyword "complete removal" is inevitable and is the most 
important. The second one "automation" has two meanings, that is, safety and efficiency. 
In the conventional research, detection and removal of mines are considered as different 
works, and the removal is after the detection. However, in the case of the excavation-type 
demining robot, detecting work will be omitted because the robot disposes of all mines in 
the target area. As the result, no error caused in the detecting work brings the demining 
rate near to 100%. 

The conceptual design of the robot is shown in Fig. 2. The robot uses crawlers for the 
transfer mechanism because of their high ground-adaptability. The robot has a large 
bucket on its front. A mine crusher is inside the bucket, and a metal separator is in its 
body. The first process of demining is to take soil into the body using the bucket. Figure 3 
shows the excavating force on the contact point between the bucket and ground. Torque T 
is generated at the base of the bucket when the bucket rotates. The torque T generates 
force F t against the ground. The body generates propelling force F v . As the result, contact 
force F is generated as the resultant force. The rotational direction of the bucket decides 
the direction of the contact force F. Therefore, the robot can realize both upward motion 
and downward motion by adjusting the bucket torque T and the propelling force F v . 
Furthermore, the robot can advance while maintaining a target depth by using some 
sensors. 

The next process is to crush mines. The soil is conveyed into the bucket by the conveyor 
belt 1 in Fig. 2. As the soil is immediately carried, the strong propelling force of the 
body is not required. The soil, which includes mines, is crushed by the crusher. Most of 
the blast with the crush escapes from the lattice 4 because the fore of the bucket is 
underground when demining. The crusher and the bucket are hardly damaged because 
the explosive power of APMs is so weak to the metal. The sufficient thickness of the 
steel plate is about 1 cm (Geneva International Centre for Humanitarian Demining, 
2002). 

The last process is to separate metal splinters of mines from the soil using a metal separator. 
Crushed debris are conveyed by the conveyer belt 2 in Fig. 2. The metal splinters, which are 
used for recycling, can be selected by an electromagnet. The rest are discharged from the 
rear. 
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Fig. 2. Conceptual design of the robot. 
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Fig. 3. Excavating force on the contact point. 



The merits and some supplementary explanations of this mechanism are as follows: 

1. This mechanism can cope with all types of mines irrespective of the size, form, and 
material of the mine. 

2. After a series of processes, the area is available for farm use immediately as the soil 
becomes clean and tilled. 

3. If the size of the lattice 4 is proper, uncrushed mines cannot go outside through the 
lattice. The uncrushed mines escaped from the bucket will be few because the 
clearance between the bucket and the ground is narrow and the blast will brow 
through the lattice. The mines will not scatter in the distance, and they will be taken 
into the bucket again in a short time. 

4. The number of ATMs (anti-tank mines) is much smaller than that of APMs (anti- 
personnel mines). It is relatively easy to search for ATMs because they are larger and 
include more metal parts than APMs. In addition, ATMs are easy to handle because 
they are difficult to ignite. ATMs are detected in advance of being taken into the 
bucket by the several sensors: metal detectors or ground penetrating radars attached 
in front of the bucket. If the robot detects ATMs, it stops before them, and the work is 
restarted after disposing the ATMs with another method. We are now considering 
the details of the treatment. 

3. Kinematics and Trajectory Planning 

The coordinate system of the robot is shown in Fig. 4. The origins of the coordinate system 
£ M and S m2 are the same. 

Xy : Coordinate system relative to the ground, 

X b : Coordinate system relative to the body, 

L bi : Coordinate systems relative to each corner of the body, 

E m : Coordinate system relative to the bucket, 

X mi : Coordinate systems relative to each corner of the bucket, 
for i = l,2,-,4. 

Each variable and constant are as follows: 

P* 2 = ( x* 2 , y*2 , ^*2 ) : Vector from the origin in coordinate system Z +1 to that in 
coordinate system £* 2 . 



Feasibility Study on an Excavation-Type Demining Robot "PEACE" 



485 



l 6* 2 ' Angle from the x-axis in coordinate system Z*j to that in coordinate system E* 2 . 
* l L? 2 • Constant length from the origin in coordinate system E*j to that in coordinate system 
£*2 . The length is the value of *3 -axis, that is x or z , in coordinate system I, n . 
The following coordinates are derived by calculating the homogeneous transform from 
coordinate system Xy to coordinate system E m4 . 

n2 )- m2 L z m4 sm(f0 b+ bl m2 ) + b L x bl cos f b - b L z bl sin f 6 b + f x b , (1) 



fx m4 = m2 Ll 4 cos(f0 b+ bl 6 



f z m4 = m2 Ll 4 sm{f0 b+ bl ml ^ m2 L z m4 ^{f0 b+ bl ml ) + b L x bl sin ' b+ b Lycos' b+ fz b . 



(2) 



From eq. (2), the control angle of the bucket bl m2 is derived as eq. (3), where the height of the 
robot f z can be measured by using some sensors like GPS and the inclinational angle of the 

body * Ojj can be measured by using a clinometer. Therefore, the target angle of bl 6 m2 can be 
calculated if the height of the end of the bucket ^ 4 is given as the target value. For example, at 

the beginning of digging, the sign of z m4 is minus, and it is constant when the robot advances 
while maintaining a target depth. The body position x b can be derived as eq. (4) by substituting 
bl 6 m2 in eq. (3) for eq. (1). The traveling body velocity f h is the derivation in time of eq. (4) and 
the velocity can be calculated if the bucket velocity f x m4 is given as the target value. 




Fig. 4. Coordinate system and parameters. 
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Fig. 7. Simulation of the excavation motion. 
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Next, the trajectory of the bucket is discussed. The robot starts to dig by lowering its bucket 
while proceeding, and it descends the slope that is made by the bucket. The target shape of 
the slope based on a cubic polynomial is shown in Fig. 5. That is the trajectory of the end of 
the bucket. The target depth was 50 cm and the slope was generated for 20 s, and then it 
went ahead while maintaining a target depth. The simulation result of the whole process is 
shown in Fig. 6, and the time response of the bucket angle hx e ml is shown in Fig. 7. The 
bucket angle does not change smoothly after about 12 s because the body tilts while it 
descends the slope. 



4. Optimal Depth of Excavation 

Generally, APMs are laid on the surface of the ground from 1 cm to 2 cm in depth (Shimoi, 
2002). However, it is possible that they are buried in the ground by deposits. It is true that 
deep excavation leads to safety, but the depth beyond necessity is not realistic from the 
aspect of working hours and cost. In this section, we discuss which depth is appropriate. 
We assumed the following: The ground is an elastic plate of the semi-infinite. Uniformly 
distributed load q is taken on a rectangular plate that is put on the surface of the ground. 
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Then normal stress to vertical direction a z , which passes through the center of the plate, is 
calculated using the theoretical formula of Frohlich , 

v L B y 

. v( i z n n ^2^,2^2,-i- 1 ^, (5) 



IkC 



In J x= ^J v= z^ 



dxdy 



where V is the stress concentration factor, z is the depth from the surface of the ground, L 
and B are the length and width of the rectangular plate respectively. The value of 
V depends on the elastic property of the soil, and it is appropriate that V = 3 is for clay soil 
and V = 4-5 is for sand deposit. 

In this study, we examined the earth load in the ground to verify eq. (5). At first, standard 
sand, of which particle size was about 0.2 mm, was put into a poly container by 20 cm in 
depth. The capacity of the container was 300 1, and the diameter and the height were 87 cm 
and 70.5 cm respectively. Then the earth pressure gauge was put on the center of the surface 
of the soil. The maximum load of the gauge was 2 kgf/cm 2 . Next, some soil was deposited 
on it and was hardened softly and evenly, and then the earth load was measured when a 
test subject put weight quietly on the rectangular plate in his one foot. The test subject was a 
man whose weight was 60 kg. The rectangular plate was wooden, and the size was 9 
cmX 22.6 cm and the thickness was 1.2 cm. The area of the plate was based on that of the 
shoe of 26 cm. We measured the earth load each five times about 10 cm, 20 cm, 30 cm, 40 cm 
and 47.7 cm in depth, and regarded each average as the representative value. 
The result was shown in Fig. 8. The closed circle in Fig. 8 represents the earth load without 
additional weight, while the open circle represents the earth load when the test subject put 
weight on the surface. The dashed line was based on the least squares approximation of the 
earth load without additional weight, while the continuous line was calculated by the 
theoretical formula of Frohlich eq. (5) when V equals 5. The dotted line represents the 
ignition pressure of PMN, Type72, MD82B and PMN2, which are the representative APMs. 
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Fig. 8. Soil pressure ( V =5). 

In Fig. 8, for example, a straight line that passes through the point of 30 cm in depth 
crosses with a line and a curve at about 0.1 and 0.15 kgf/cm 2 respectively. In this case, the 
pressure only of the soil is 0.1 kgf/cm 2 , and it changes to 0.15 kgf/cm 2 when the test 
subject puts weight on the surface. The mines of which ignition pressure is less than 0.1 
kgf/cm 2 hardly remain unexploded because almost all of them explode under the earth 
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load. The mine of which ignition pressure is from 0.1 kgf/cm 2 to 0.15 kgf/cm 2 explodes 
when the test subject puts on it. The mine of which ignition pressure is more than 0.15 
kgf/cm 2 , however, does not explode with the test subject who weighs 60 kg because the 
ignition pressure is over the total load of the soil and him. Summarizing the above, there 
are few mines in the area of 1. Mines are not active in the area of 2. Mines will explode if 
the test subject steps into the area of 3. It is desirable that the area between 1 and 2 is 
narrow. However, deep excavation results in high cost. In addition, a certain amount of 
overburden will contribute to preventing the immense damage. Figure 9 shows the result 
to various stress concentration factors V . In case of sand: V = 5 or in case of Fig. 8, 
vertical stress <* z was highest on a certain depth. That means the environment of sand is 
the most dangerous because the area of 3 in Fig. 8 is widest. Therefore, it was confirmed 
that 40 cm was proper depth. 

Next, we discuss the dynamic load. The continuous line in Fig. 10 shows the experimental 
results when the test subject walked on the surface of the soil. The depths of the soil were 
from 10 cm to 40 cm. The dashed lines show the experimental results for the static loads. 
From this result, we can see that the load on the surface did not influence the underground 
so much when the depth was over 30 cm. 
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Fig. 9. Relationship between the vertical stress and V . 
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Fig. 10. Time response of the soil pressure when walking. 
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Fig. 11. Position identification. 



In conclusion, we propose 40 cm as the best excavation depth. This depth is also valid for 
farmland. In case mines are buried in a leaning position or that the soil has solidified with 
passing years, the mines become difficult to explode. Therefore, the proposed excavation 
depth is safer. 



5. Hardware of the Scale Model 

In order to control the robot autonomously, it is necessary to measure the position of the 



robot ( J x b , J y b , J z b ). In the actual robot, it will be measured by using GPS. High-precision 
GPS (RTK-GPS) is, however, expensive and difficult to use indoors, so a high-precision and 
inexpensive positioning sensor was produced in this study. 

The positioning sensor is a landmark system. Three landmarks Pi, P2, P3 are set above the y- 
axis (0, -A , C), origin (0,0,C), x-axis ( — B, 0, C) respectively as Fig. 11, and the heights of 
them are the same. Each distance from the robot is r i ( i =1,2,3). The angles a and /3 are the 
angular errors that are caused when installing the positioning sensor units. The position of 
the robot P (x,y ,z) is derived as follows: 



sin/? cosar 
cos J3 sin a 



B( n 2 -r 2 2 -A 2 ) 
A(r 3 2 -r 2 2 -B 2 \ 



n I 2 2 2 

z = C-^r 2 -x -y 



where 



The sign before the root in eq. (7) is negative if the landmark is above the robot. 



(6) 
(7) 
(8) 
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Fig. 12. Positioning sensor unit. 
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Fig. 14. Excavation type mine removal robot. 
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Fig. 15. Time response of the wheel angle. 
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Fig. 16. Body position. 




Fig. 17. Time response of the body angle and bucket angle. 
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Fig. 18. Time response of the bucket position. 
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Figure 12 shows a photograph of one positioning sensor unit. A fishing line is wound round 
a pulley connected to an encoder. The line is stretched tight because a fixed voltage is 
applied on a motor that winds the line. It is possible to measure the position P (x, y, z) by 
using three sensor units in theory. In that case, however, it was clarified that large errors on 
z-axis occurred through experiments. The measure for that is to attach one more sensor unit 
on z-axis. 

To estimate the accuracy of the positioning sensor on the x-y plane, positions of lattice points 
in the area of 1.5 m square were measured in every 0.25 m. The result is shown in Fig. 13. 
The maximum error was approximately 5.56 cm. After corrections about a , j3 and the 
diameter of the pulley connected to each encoder, the maximum error was approximately 
0.69 cm. 

The model of the excavation-type demining robot has a scale of 1 to 10. The dimensions of 
the robot are 0.36 m X 0.84 m X 0.217 m in height. The photograph is shown in Fig. 14. The 
crawlers for the transfer mechanism are the parts of a model tank. Each crawler can be 
rotated independently by a motor with an encoder. The inclinational angle of the body was 
measured by a fiber optical gyroscope (JG-108FD1, Resolution < 0.01 deg, Frequency 
response 20 Hz; Japan Aviation Electronics Industry, Ltd.). The bucket angle was measured 
by a clinometer (NG3, Resolution < 0.003 deg, Frequency response > 3.3 Hz; SEIKA 
Mikrosystemtechnik GmbH) and a potentiometer; the former was used for monitoring the 
bucket angle and the latter was for control. The soil in the bucket was carried by a conveyor 
belt to the body and discharged backward of the body without being crushed. The robot 
was controlled by a PD controller. 

Figures 15-18 show the experimental results. The robot went through 50 cm distance in 20 s. 
The bucket reached the target depth of 3 cm in 10 s and kept it afterward. The measured 
data of the wheel angle almost agreed with the target. Figure 15 shows the time response of 

the wheel angle. Figure 16 shows the position of the robot ( * x b , * z b ) that was measured by 
using three positioning sensor units shown in Fig. 12 installed with tilting 90 deg. The errors 
were mainly caused by the unevenness of the ground and the slippage of the crawler. Those 
errors can be reduced if the measured position of the robot is programmed in the feedback 
control loop. Figure 17 shows the time response of the bucket angle measured with the 
clinometer and the inclinational angle of the body. The body was lifted slightly by the drag 

from the bucket. Figure 18 shows the bucket position ( * x m4 , * z w4 ) that was calculated from 

eqs. (1) and (2) based on the bucket angle measured with the clinometer. The shape of it 
resembled Fig. 5. The vibration of the target trajectory was caused by the measured position 
and inclinational angle of the body. The bucket followed the trajectory and dug to near the 
goal depth. 

6. Crush Process 

The processing reliability of demining machines or robots is the most important to humanitarian 
demining. The main point is the crush process. Most conventional rotor-type demining machines 
have rotor bits that are larger than mines. Therefore, it is possible that the mines do not touch the 
bits and they are left unprocessed. In addition, high reliability cannot be realized without the 
sifting process. Almost all of conventional machines, however, have no such process. Although 
MgM Rotar has this process, the demining efficiency is not so high (Geneva International Centre 
for Humanitarian Demining, 2002). We discuss the structure of the crusher. We made the test 
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equipment shown in Figs. 19 and 20 in order to examine the validity of it and the improved points. 
The characteristics are as follows: 

1. The rotational frequency of the rotor is rapid so that the debris may become small. 

2. Small rotor bits are mounted around the rotor in small intervals, and every line is 
mutually different. 

3. The plate shown in Fig. 19 is installed under the rotor of which direction of rotation 
is anticlockwise. 

Because of the characteristics 2 and 3, sifting process is realized. 

The width of the test equipment was 554 mm, the length was 450 mm, and the height was 
280 mm. The diameter of the rotor was 100 mm without bits, and its width was 207 mm. 
Setscrews of M5 were used for the bits. Each lateral interval of bits was 10 mm, and twenty- 
four bits were mounted around the circumference of the rotor. The rotor was driven by DC 
motor of 250 W, and its rotational frequency was measured by an encoder. The electric 
current supplied to the motor was also measured. A box made of the acrylic covered the 
equipment and was used to prevent some debris from being scattered. A wheeled mobile 
robot shown in Fig. 20 carried the experimental samples on the plate to the rotor that was 
fixed on the base. 




Fig. 19. Test equipment for crush (top view) 




Fig. 20. Test equipment for crush (side view). 



494 



Mobile Robots, Towards New Applications 



As the experiment samples, we used clods of the loam layer, in which there is nothing 
explodable. The amount of the clods for one experiment was 160 g. It is true that the samples 
were more easily broken than the real earth but they were enough to examine the 
phenomena. The electric current and particle sizes of the samples after the crush were 
measured according to various conditions: the rotational frequency of the rotor, the length 
of the bits, the gap between the tip of the bits and the plate fixed under the rotor, and the 
traveling speed of the robot. They were varied on condition that each standard value was 
2100 rpm, 5 mm, 2 mm and 10 mm/s respectively. The particle sizes of the samples after the 
crush were sorted out at four groups: 0-3 mm, 3-7 mm, 7-10 mm, and over 10 mm. The 
lengths of 3 mm, 7 mm and 10 mm correspond to the hole size of the square lattice plates 
used for sorting. 
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Fig. 21. Comparison for the rotational frequencies. 
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Fig. 22. Comparison for the rotational frequencies. 

Figures 21 and 22 show the experimental results to each rotational frequency of the rotor: 
2100 rpm, 925 rpm and 740 rpm, which was changed by speed reducers. The voltage 
supplied to the rotor motor was constant. In the conventional demining machines, the 
frequency is at most as 700 rpm (Geneva International Centre for Humanitarian Demining, 
2002). Figure 21 shows that high rotational frequency leads to continuous crush. Figure 22 
shows that high rotational frequency is related to fineness of the particle sizes of the samples 
after the crush, that is, high processing reliability. 
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Fig. 23. Comparison for the lengths of the bits. 
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Fig. 24. Comparison for the lengths of the bits. 

Figures 23 and 24 show the experimental results to the length of the bits. Each length of the 
bits was 5 mm and 15 mm except for the length of the mounting parts that was about 5 mm. 
Both results were almost equal because of the high rotational frequency of the rotor. In case 
of 15 mm bits, however, there were some debris that passed through the 10 mm square 
lattice. While, in case of 5 mm bits, no such debris were left. 
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Fig. 25. Comparison for the gaps. 
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Fig. 26. Comparison for the gaps. 

Figures 25 and 26 show the experimental results to the gap between the tip of the bits and 
the plate fixed under the rotor. Three kinds of gaps were examined: 1 mm, 2 mm and 3 mm. 
The smaller the gap became, the smaller the samples were crushed and the electric current 
of the motor slightly increased. Therefore, it was confirmed that small distance is desirable 
for humanitarian demining operation. 




Fig. 27. Comparison for the traveling speeds. 
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Fig. 28. Comparison for the traveling speeds. 

Figures 27 and 28 show the experimental results to the three kinds of traveling speeds of the 
robot: 10 mm/s, 15 mm/s and 20 mm/s. As the robot traveled faster, the maximum value of 
the electric current of the motor increased. However, there is little difference to the particle 



Feasibility Study on an Excavation-Type Demining Robot "PEACE" 497 

sizes of the samples after the crush. Therefore, in respect of time efficiency, it is desirable 
that the robot travels fast within the limits of keeping the rotational frequency of the rotor. 

7. Summary and Future Works 

In this study, we proposed a novel excavation-type demining robot that worked 
autonomously. The advantages of the robot are a high clearance capability and high 
efficiency. The crusher inside of the robot plays two roles: crushing the mines and sifting 
soil. Therefore, the robot has a high clearance capability. It also has a mechanism for 
separating metal splinters of mines inside. In addition, the robot can perform a series of 
those operations continuously. So it has high efficiency. 

However, this robot is not all-powerful for all environments. The main target area for our 
robot is the place that becomes the farmland after demining. For example, the areas are 
plains or past farmland. The reason why we choose farmland as the demining area is that 
farmland is such an area where local people cannot help entering to live, and so it should be 
given the highest priority. For severe conditions, it will be necessary to cooperate with other 
types of machines, robots and some sensors systems. 

The kinematics and motion planning for the robot were discussed. The possibility of demining 
was verified by a scale model of the robot. It was confirmed that 40 cm was the proper depth of 
excavation through experiments. The crush process that had high reliability was also discussed. 
The implementation of a prototype is as follows: the power source for the robot is hydraulic, the 
required minimum shielding is 10 mm steel plate, and the approximate size and weight are 3.6 X 
8.4 X 3.0 m in height and 50 t respectively. We assume that the maximum crushable rock size is 
about 20 X 20 X 20 cm. If a bigger and sturdy rock comes into the bucket, the rotor may stop. In 
this case, the robot will have to stop the operation and to remove the rock. This is our future work. 
For the navigation system, we use some sensors as follows: the global position of the robot is 
measured with RTK-GPS. The tilt angles of the body (roll and pitch angles) are measured with 
clinometers. The orientation angle of the body (yaw angle) is measured with a gyroscope. Stereo 
video cameras are necessary to measure the corrugation of the ground and to monitor the ground. 
Laser range finders and ultrasonic sensors are necessary to find the obstacles. 
It will need a lot of money to develop and maintain the robot. However, we are convinced that our 
robot is one of the best solutions concerning a clearance capacity and efficiency. We wish the robot 
would be provided by an advanced nation as a part of a national contribution. 
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1. Introduction 



Unlike ground-base robot manipulator, the space manipulator has no fixed base. The 
dynamic reaction forces and moments due to the manipulator motion will disturb the space 
base, especially, when the space robot is in free-floating situation, The longer the motion 
time of space manipulator is, the greater the disturbance to the base will be. Hence, it is 
essential to resolve the attitude balance problem of a space robot during the manipulator 
operation. The position disturbance of the base may not be an issue. The attitude change, 
however, is more serious because the solar panel and tele-command from the ground station 
should keep their life lines. On the other hand, the attitude changes possibly cause the 
serious collision between the manipulator hand and the target. Therefore, the attitude 
balance problem of free floating space robot is a challenge problem about the application of 
space robot in space operation. 

Nowadays, space robots are aiding to construct and maintain the International Space 
Station (ISS) and servicing the space telescope. Therefore, space robotic servicing for 
satellite such as rescue, repair, refuelling and maintenance, promising to extend 
satellite life and reduce costs, made it one of the most attractive areas of developing 
space technology. Space robots are likely to play more and more important roles in 
satellite service mission and space construction mission in the future. A well known 
space robot, the Shuttle Remote Manipulator System (SRMS or "Canadarm") (D. 
Zimpfer and P. Spehar, 1996) was operated to assist capture and berth the satellite 
from the shuttle by the astronaut. NASA missions STS-61, STS-82, and STS-103 
repaired the Hubble Space Telescope by astronauts with the help of SRMS. 
Engineering Test Satellite VII (ETS-VII) from NASDA of Japan demonstrated the space 
manipulator to capture a cooperative satellite whose attitude is stabilized during the 
demonstration via tele-operation from the ground base controln station (I. Kawano, et 
al. 1998), (Oda, M.1999), (Oda, M., 2000), (Noriyasu Inaba, et al, 2000). The space 
robotic technologies mentioned above demonstrated the usefulness of space robot for 
space service. During the ETS-VII operation, the engineers considered the coordinated 
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control of the manipulator and space base in order to compensate the reaction forces 
and moments. Obviously, compensating the attitude disturbance of space base will 
consume much fuel during the motion, at the same time, the fuel is very limit and not 
reproduced. The life of space robot will be reduced by a long way. Therefore, the 
coordinated control of manipulator and space base is not the best method to control 
the space robot. 

There are many studies on dynamic interaction of space base and its manipulators. 
Dubowsky and Torres (Steven Dubowsky, Miguel A. Torres, 1991) proposed a method 
called the Enhanced Disturbance Map to plan the space manipulator motions so that the 
disturbance to the space base is relatively minimized. Their technique also aided to 
understand the complex problem and developed the algorithm to reduce the disturbance. 
Evangelos G. Papadopoulos (Evangelos G. Papadopoulos, 1992) exhibited the 
nonholonomic behavior of free-floating space manipulator by planning the path of space 
manipulator. They proposed a path planning method in Cartesian space to avoid 
dynamically singular configurations. Yoshida and Hashizume (K.Yoshida, et, al, 2001) 
utilized the ETS-VII as an example to address a new concept called Zero Reaction Maneuver 
(ZRM) that is proven particularly useful to remove the velocity limit of manipulation due to 
the reaction constraint and the time loss due to wait for the attitude recovery. Moreover, 
they found that the existence of ZRM is very limited for a 6 DOF manipulator, besides the 
kinematically redundant arm. However, the redundant manipulator gives rise to the 
computational cost and complicated kinematics problems. Hence, it is not best way to use 
the redundant manipulator from the view of engineering point. 

Nearly all examples mentioned above almost focused on the nature characteristics of 
the space robot with a single robot arm. Moreover, these previous research largely 
limited the application of manipulator so that the space manipulator can not do 
desired mission like as ground-base manipulator. Thus, the space manipulator's 
motion must follow the constrained path in order to minimize the disturbance to the 
space base. Therefore, the manipulator can not dexterously be operated to accomplish 
the desired mission, i.e, if the manipulator tracks the desired mission trajectory, the 
disturbance to the base can not be minimized. Hence, it is very difficult to minimize 
the disturbance or ZRM for desired capture mission. Fortunately, an obvious 
characteristic of the space robot with respect to ground-base robot is the dynamics 
coupling between the robot and it's base. Therefore, the dynamics coupling provides 
some potential methods for researchers and engineers. Using the dynamics coupling, 
we will address a novel methodology to compensate the disturbance other than the 
attitude control system (ACS) in this chapter. 

In general, the dynamics coupling may be a disadvantage for space robot system, many 
previous research work focus on how to eliminate it. On the other hand, the dynamics 
coupling may be a advantage if we use it correctly. Using the nonholonomic path planning 
method to control robot motion, the desired attitude of the base can be maintained as 
proposed by Fernandes, Gurvits, and Li (C. Fernandes, et, al, 1992) and Nakamura and 
Mukherjee (Robert E Roberson et, al 1988), and the methods will be best implemented to a 
system with a strong dynamics coupling. Therefore, it is necessary to fully understand the 
dynamics coupling, and try to eliminate it completely, rather than to minimize the coupling 
effect at the beginning. The measurement of dynamics coupling was firstly addressed by Xu 
(Yangsheng Xu et, al, 1993). 
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In this chapter, we discuss the problem of dynamic balance control of a space robot 
when space manipulator follows the specified path to capture the object, rather than use 
ACS to compensate dynamics coupling. Therefore, we proposed to use a balance arm to 
compensate the dynamics coupling due to mission arm's motion, thus, our idea changes 
the disadvantage of dynamics coupling as advantaged factor. Accordingly, this chapter 
is organized as follows. We formulate the dynamics equations of multi-arm free-floating 
space robot, based on linear and angular momentum conservation law and Lagrangian 
dynamics in Section II. In section III, we analyze and measure the dynamics coupling 
between the base and its multi-arm manipulators. Then, we address dynamic balance 
control algorithm of the base during the motion of manipulators in Section IV. In 
Section V, the computer simulation study and result are shown. Final section 
summarizes the whole chapter and draws the conclusion. 

2. Multi-Arm Space Robot System 

2.1 Modeling of Multi-arm Free-Floating Space Robot 

In this section, we describe the model of multi-arm space robot. These arms are installed 
on a base (spacecraft or satellite), one of these arms is called Mission Arm which is used 
to accomplish the space mission, such as, capturing and repairing, the others are the 
balance arms which are used to balance the reaction motion due to the motion of the 
mission arm. Motion planning and control of the balance arms are our key problems 
which will be introduced in the following sections. Here, the space robot system is in 
free-floating situation during operation in order to avoid the serious collisions and save 
the fuel. Therefore, no external forces or momentum are exerted on the space robot 
system. The linear and angular momentum is conserved in this situation. 
The authors assume a model of multi-arm space robot system which is composed of a 
base and multiple manipulator arms mounted on the base. Fig. 1 shows a simple model 
of the space robot system with multi-arms, which is regarded as a free-flying serial 
nm+1 link system connected with nm degrees of freedom (DOF) active joints. Here, in 
order to clarify the point at issue, we make the following assumptions: (a) The each 
installed manipulator consists of n links, and each joint has one rotational degree of 
freedom and is rate controlled. The whole system is composed of rigid bodies. 
Moreover, there are no mechanical restrictions and no external forces and torques for 
simplicity, so that linear and angular momentum is conservative, and equilibrium of 
forces and moments, strictly hold true during the operation; (b) the kinematics and 
dynamics analysis during the motion is in the inertial coordinate system; (c) the DOF 
of space robot system in inertial coordinate system is nm+1. The attitude of satellite 
has three DOFs, and the translation of the satellite has three DOFs. Here, n represents 
the number of robot arm and m represents the DOF number of each robot arm. 
As shown in Fig. 1, the dynamics model of space robot can be treated as a set of nm+1 
rigid bodies connected with nm joints that form a tree configuration, each manipulator 
joint numbers in series of 1 to m. We define two coordinate systems. One is the inertial 
coordinate system S| in the orbit, the other is the base coordinate system 2 B attached 
on the base body with its origin at the centroid of the base. The COM is the center of 
total system mass. We use three appropriate parameters such as Roll, Pitch, and Yaw 
to describe the attitude of the base. 
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Manipulator n 



Manipulator 1 , V. 




Space Robot Base 

Fig. 1. Dynamics Model of Space Robot System. 



2.2 Equation of Motion 

In this section, the equation of motion of a multiple-arm free-floating space robot system are 
obtained according to proposed dynamics model. The dynamical influence due to orbital 
mechanics are neglected because of relative short length and duration of the travel of system. 
It is a general problem to derive the dynamics equations of space robot system. We use 
Roberson-Wittenburg method [Robert E Roberson et, al 1988] and previous work [E. 
Papadopoulos, et, al 1990] to derive the rigid dynamics of multi-body system. This method 
utilizes the mathematical graph theory to describe the interconnecting of the multi-body. 
The advantage of this method is that the various multi-body systems can be described by 
the uniform mathematical model. Thus, we will simplify describe the derivation process of 
dynamics equation according to the assumed model.The total kinetic energy of space robot 
system with respect to inertial coordinate E ( can be expressed as follows. 



with 



T 1 = -{m R B z +nolo^o)+^ l^L 

li=i j=i 



T=Tr 



To=2 Mv CM 



n k 



l! ^^( m (i.j/(u 




(1) 



(2) 



(3) 



where I (j , j > is the inertial matrix of ;-th body of i-th manipulator arm with respect to system 
COM. Therefore, the kinetic energy of whole system can be rewritten as. 



T =^ vT|H (Xo.# 



(4) 



where v = IX ,9 j is the vector of generalized velocities, and X = (xJ f ^ T )i s the vector to 
represent the position and orientation of the base, and matrix H ( x ,$) is an positive definite 

mass matrix, and ^ = (^...,^...,^,...4") . 
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From the kinetic energy formulation, we can derive dynamics equation by Lagrangian 
dynamics when there are not external forces or momenta exerted on the space robot system. 

H V + C^) = r (5) 

where H * is a generalized inertial matrix and C * is a centrifugal and coriolis term. 

1 H 0m 



H* 



H 0m H m 



(6) 



C* = HV-fr|-=f HV| (7) 



-M 



(8) 



The symbols in the above equations and Fig. 1 are defined as follows: 
m(ji) ' mass of link; of the z-th manipulator arm 

f(j f j ) : position vector of centroid of link; of the z-th manipulator arm in E 1 system 

R(j i \ : position vector of joint; of z-th manipulator arm in E B system 

P(i,e) : position vector of end-effector of z-th manipulator arm in coordinate system origin in 

E B system 
R(i,e) : position vector of end-effector of z-th manipulator arm in coordinate system origin in 

E B system 
R B : position vector of centroid of the base body in E 1 system 
r g : position vector of a total centroid of the space robot system in E, system 
I (i j \ : inertia tensor of link; of z-th manipulator with respect to its mass center 
T : joint torque of the whole manipulator arms 
(j) : joint variables and base variables 

X : position/ orientation of the base 
H o '> the inertial matrix of the base 
H m : the inertial matrix for the manipulator arms 

H om : coupling inertial matrix between the base and robot manipulator 

All vectors are described with respect to the inertial coordinate system E| and body 
coordinate system E B . In Equation (5), C* can be obtained by inverse dynamics 
computation. 

3. Dynamics Coupling 

The dynamics coupling may not always be a disadvantage for the space robot system, as it 
can be used to balance the attitude of the base sometimes. In this chapter, we will introduce 
the balance concept to compensate the attitude disturbance using dynamics coupling. This 
section addresses the characteristics and measure method of dynamics coupling. 
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The dynamics coupling is significant in understanding the relationship between the 
robot joint motion and the resultant base motion, and it is useful in minimizing fuel 
consumption for base attitude control. The measure of dynamics coupling was 
proposed by Xu (Yangsheng Xu, 1993). Here, we will simply derive the mapping 
relationship between manipulator motion and the base motion for multi-arm space 
robot system. 
Let V(j i) and ^(u) be linear and angular velocities of the;-th body of the z-th manipulator arm 

with respect to, E| and let Vh :\ and (Oh -. \ be that with respect to 2 B . Thus, we can obtain. 



w U) = v (U) +v o + Q o xr (i,j 

Q (U) = ^(U) +Q o 



(9) 



where Vgand fig are linear and angular velocities of the centroid of the base with respect to 
2 1 . The operator x represents the outer product of R vector, the velocities 
V(j j \ and Q)n j \ in the base coordinate Eg can be calculated by following equation. 



U.J) 
'(i.J). 



= J(i.j)(^ 



(10) 



U.J) 



JT(i.j) 
jR(i.j) 



(11) 



where J (i j )(#) is the jacobian of the;-th body of the z-th manipulator arm. 

Therefore, the total linear and angular momentum of the space robot system P, L is 
expressed as follows. 



n k 



P=moV + 2Z( m (U) m (U)) 
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n k 
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(13) 



These equations are rewritten with the variables V Q/ C0 and . 



= H, 



fir 



H m 6 



(14) 



where 
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n k 



H m=XZ( l R(i,i) l (i.i) J R(i.j) + m (i.i)JT T (i,i) J T(i.J)) 



i=l j=l 



J R(i-J) = L ' — °- d (Xj)' d (2.j) d (i,j)' '"-°J 

j T(i:j) = [0,...,0,d (i , j) x(r (i:j) -d (i , j) ),0,...0J 



(16) 

(17) 
(18) 



J R /j : \ is Jacobian matrix of ;'-th joint of z-th maniulator and map the angular velocities of 
joint and angular velocity of ;-th link. As such, the J t(u) m aps the linear velocity. E is unit 
vector indicating a rotational axis of joint j of manipulator i. fy . \ is position vector of the 
mass center of link; of manipulator i, (L -\ is position vector of joint; of manipulator i. 

In the free-floating situation, no external forces or momenta exerted on the whole space 
robot system, and we ignore the gravitational force. Therefore, the linear and angular 
momenta of the system are conserved, we assume that the initial state of system is 
stationary, so that the total linear and angular momenta are zero. Hence, Equation (14) can 
be rewritten as follows. 



V 
Q 



-H m H B<? 



(19) 



Equation (19) describes the mapping relationship between the manipulator joint motion and 
the base moti on. Because the velocities of end-effector in base coordinate is related to the 
joint velocities, we can obtain. 



a (i,e) 



(i.j) 1 



e 



(20) 



However, the velocity of end-effector in inertial coordinate S, can be represented as follows. 

fVn 



IUJ 



:V (i,j)+ V + Q xr (i,e) 



Q 



(i,e) =a) (\,e) 



+ Qr 



(21) 



Combining Equation (19), (20), and (21), the relationship between the velocities of end- 
effector and the base velocities in inertial coordinate can be obtained as follows. 



V (U) 

Q (i.j) 



where 



= J(i,j)-KH m V 



' E [ R (i,e)xfi 
0*3 E 
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(22) 



(23) 



In Equation (22), let M =J(ji)-KH m H B , and the inverse relation of Equation (22) is 
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V 
Q 



= N 






(24) 



The matrix M and N describe how robot motion alters the base motion, and these two 
matrices represent the dynamics interaction between the robot manipulator and the base, 
They are known dynamics coupling factor (Yangsheng Xu, 1993). Equation (24) defines the 
interactive effect between the end-effector's motion of manipulator and the base motion. The 
dynamics coupling factor defined above basically represents the coupling of motion. 
Similarly, we also can define the coupling factor for force transmission from the end-effector 
to the base, thus we can easily obtain the force relationship as follows. 

F B =N- T F e (25) 

During the capture operation of space manipulator, the impact between the end-effector and 
the target is not avoided, so that, Equation (25) can measure the disturbance force to the base 
due to the collision. On the other hand, these mapping relationship can be used to plan 
carefully the robot manipulator motion in order to 
minimize the disturbance to the base. 

In fact, the dynamics coupling of space robot system is very important to understand the 
dynamics characteristics and design the control part of the system. Especially, in the free- 
floating situation, this coupling must be considered in order to realize the minimum 
disturbance. In the next section, we will design the balance control algorithm to compensate 
the disturbance using dynamics coupling factor mentioned above. 

4. Dynamic Balance Control 

4.1 Dynamic Balance Control Concept 

In this section, we address a balance control strategy for multi-arm space robot system. Here, we 

assume that the space robot system has a two-arm robot for simplicity, The number of DOF of 

each arm is same or not. We define these two arms as the mission arm and the balance arm, 

respectively. The mission arm is used to accomplish desired mission and the balance arm is used 

to compensate the attitude of the base using the dynamics coupling concept. 

In order to balance the attitude disturbance of the base well, the authors utilize the 

coordinated control concept (Yangsheng Xu, et, al 1994) to address the dynamic balance 

control method proposed by them. However, according to our research problems, we can 

not use the attitude control system to compensate the robot arm's reaction so 

that we use the balance robot arm to compensate the disturbance. Therefore, we address our 

balance control concept as follows. 

(a) For two-arm space robot system, since the DOF of the whole system becomes so large 
that it is difficult to handle computation and commands using the state-of-the-art 
onboard computer. Hence, we suggest that each robot arm's controllers are independent. 

(b) Because the motion trajectory of the mission arm has been planned for the desired 
operation, the reaction momenta to the base can be predicted using the dynamics 
coupling factors in advance. 

(c) We build the mapping relationship of the mission arm's motion and balance arm's 
motion in the zero attitude disturbance condition. Thus, the trajectory of the balance arm 
will be defined by the trajectory of the mission arm. 
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The balance control concept proposed by us uses the motion of the balance arm to 
compensate the attitude disturbance due to the motion of the mission arm. we will address 
the balance control algorithm in detail in the following subsection. 

4.2 Coordinated Control of Two Arms 

Coordinated control of space robot was addressed by Oda, M.,(M. Oda, 1996), he 
presented the coordinated control concept between the spacecraft and its manipulator, 
Here, we use this concept to our problem, we only consider the attitude disturbance 
of the base for free-floating space robot system. Hence, the angular momenta of the 
whole system can be estimated by Equation (13). At the beginning of the mission ram 
motion, the whole system keeps stable so that the initial momenta are zeros. 
Therefore, during the capture operation of the mission arm, we can get the 
relationship between the joint velocities of mission arm and the attitude angular 
velocities of the base from Equation (14) as follows. 

n 0=J(M ffl ))^M ( 26 ) 

Differentiate Equation (26) with respect to time, we can obtain. 

^0=J (M,^M +J (i,j)#M ( 27 ) 

where 

J (M ,^) = - H (M 1 ,m) H B (28) 

H~B=i(M^)+wr g r 0g (29) 

H (M,m) = ' (M ,^) — r g J la ( 30 ) 

However, the angular momentum given by Equation (26) can not be integrated to 
obtain the base orientation y/ as a function of the system's configuration, <9 M because of 
the nonholonomic characteristic, i.e we can not obtain the analytical solution to depict 
the relationship between the attitude I// and joint angle 6w . However, Equation (26) 
can be integrated numerically, thus, the orientation of the base will be a function of the 
trajectory of manipulator in joint space, i.e. different trajectory in joint space, with the 
same initial and final points, will generate different orientation of the base. The non- 
integrability property is known nonholonomic characteristics of free-floating space 
robot (E. Papadopouls, 1992). 

For the given capture operation, the mission arm motions along the given trajectory. 
According to the trajectory planning method and dynamics equations, we can get the 
joint angle, angular velocities, and angular acceleration of the mission arm in joint 
space. Therefore,we can obtain the angular velocities, and acceleration of the base 
using Equations (26), (27) as mentioned above. On the other hand, we also can 
numerically integrate Equation (26) to obtain the orientation function of the base with 
respect to the joint variable of the mission arm. Therefore, we can obtain the torque 
exerted on the base because of the motion of the balance arm. Generally speaking, the 
attitude control system drives the reaction wheels to counteract this torque in order to 
balance the attitude of the base, our concept is to utilize the balance arm to substitute 
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for the reaction wheels, an important advantage of this method is to save the fuel that 
is limit and valuable in space. 

Hence, we use the motion parameters of the base due to the mission arm's motion to 
calculate the joint angle, joint angular velocities and joint acceleration of the balance arm. 
These parameters of the balance arm consist of the balance arm's trajectory in joint space. To 
realize the dynamic balance of the base's attitude, the attitude changes of base due to 
balance arm's motion (p should satisfy y/ + (p = 0, so that the attitude disturbance of the 

base almost be zero. Thus, We can obtain the following two equations to represent the 
relationship between the balance arm and base's attitude. 

^B =J(B 1 «)"0 (31) 

»B="0-J'(B^B (32) 

where 0% is the joint variables of balance arm, and J ( B iC0 \ is the Jacobian matrix of balance arm. 

For numerical simulation, Equation (31) can be integrated numerically to obtain the joint 
angle trajectory of the balance arm. Thus, we can get the desired trajectory of the balance 
arm in joint space, the balance arm's motioning along this desired trajectory can dynamically 
balance the attitude of the base. Thereby, our balance control concept can be realized. The 
effectiveness of balance control algorithm can be demonstrated in the simulation study of 
the following section. 

5. Simulation Study 

To better understand and examine the practical balance control performance of the 
proposed method, we define a two-arm space robot system as an illustrative example to 
attest the balance control method. In simulation, we assume that the object is in the 
workspace of the mission arm for simplicity. We have carried out relatively precise 
numerical simulation with a realistic 3D model. 

The model comprises a two-arm manipulator which is called mission arm and balance arm, 
respectively. Each arm has 3-DOF, and every joint is revolution joint, Moreover, the 
geometric structure of two-arm manipulators is same in order to simplify the computation. 
Here, we assume that each link of manipulator is cylindric configuration. The radius of link 
r =0.04:m, and the base of space robot system is cubic configuration. Table I shows the 
dynamics parameters of the two-arm manipulator space robot system. 

In simulation, we firstly plan the mission arm trajectory using Point to Point algorithm (PTP) 
so that the mission arm can approach and capture the target. However, the orientation of 
end-effector of the mission arm will be effected because the dynamics coupling in free- 
floating situation so that the collision between the end-effector and target becomes very 
serious. Therefore, the dynamic balance control concept can compensate this disturbance 
using the disturbance of motion of the balance manipulator. From the academic point of 
view, this method is very useful, the simulation result can verify our balance control 
algorithm. The balance control concept can be divided into two phases, 

Phase I mainly computes dynamics coupling and base motion trajectory because of 
disturbance of master manipulator motion in free-floating situation. Phase II calculates the 
required joint trajectory of the balance arm according to the estimation of Phase I. We will 
address these two phases in the following parts in detail. 



Attitude Compensat 


ion of Space Robots for 


Capturing Operation 






509 






Space 

Base 

(Link 0) 


Space Manipulators 


Mission Arm 


Balance Arm 
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Link2 


Link3 


Linkl 


Link2 


Link3 


Mass (Kg) 


300 


5.0 


5.0 


55 


5.0 


5.0 


55 


Length( m ) 


1.0 


0.5 


0.5 


0.5 


0.5 


0.5 


0.5 


1 xx (Kg-m 2 ) 


50 


0.4187 


0.004 


0.0044 


0.4187 


0.004 


0.0044 


1 yy (Kgm 2 ) 


50 


0.4187 


0.1062 


0.1168 


0.4187 


0.1062 


0.1168 


l zz (Kg-m 2 ) 


50 


0.16 


0.4187 


0.4606 


0.16 


0.4187 


0.4606 



Table. 1. Parameters of Space Robot System. 

Phase I: According to the planned joint trajectory of missionmanipulator, we design the PD 
feedback controller to drive the joint. We use the inverse dynamics to calculate the dynamics 
coupling due to the motion of mission manipulator, at the same time, we also can obtain the 
changes of orientation, angular velocities, and angular accelerative velocities of the base 
because of motion of the mission arm. Here, we use the precise numerical simulation so that 
we can integrate numerically Equation (26) to obtain the mapping relationship between the 
base orientation and the mission manipulator's configuration. 

Phase II: We calculate the desired trajectory of the balance arm according to the motion 
trajectory of the base so that the disturbance of the balance arm's motion can compensate the 
disturbance of the mission arm's motion, i.e. combining the negative influence due to the 
mission arm's motion with the positive influence due to the balance arm's motion equals to 
zeros, Hence, the balance control method is realized. 

For the simulation result, Fig. 2 and Fig. 3 depict the attitude disturbance of the base because of the 
motion of the mission manipulator and that of the balance arm respectively. Fig. 4 shows the 
reaction resultant torque to the base after using balance control algorithm. Fig. 5 depicts the 
attitude disturbance after using the balance control algorithm. Note that the attitude disturbance of 
the base becomes very small at the end of the capture operation, as shown in Fig. 5. 
The goal of simulation is to verify that dynamic balance control method proposed is right 
and useful. The authors also measure the dynamics coupling between the mission arm and 
the base in order to confirm whether the balance arm can balance this coupling and 
compensate the orientation disturbance of the base. 
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Fig. 2. Attitude changes of the base due to the mission arm motion. 
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Fig. 3. Attitude changes of the base due to balance arm motion. 



0.08 

0.06 

0.04 

^ 0.02 

I 

| -0.02 

^ -0.04 

I -0.06 

-0.08 

-0.1 

-0.12 



Reaction Resultant Torque after Balance 


Control 


"""! -]--t-;-t--y-|--]-- t 


X direction 

— Y direction ■ 
Z direction 






! ! / ! \! Ua7\[ ! 


i^d-.. ! ! i /f 7 ! s A\i 


| j ]4---W--j ( | 


! ! ! i\""1 \ 1 ! ! ! 


i i ! i i i i i i 



15 20 25 30 35 40 45 50 
time(s) 



Fig. 4. Reaction resultant torque after balance control. 
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Fig. 5. Attitude changes after balance control. 



Attitude Compensation of Space Robots for Capturing Operation 51 1 



6. Conclusion 

In this chapter, we studied the dynamic balance control of multi-arm free-floating space 
robot. According to unique characteristics of free-floating space robot, we presented the 
dynamics coupling representing the robot arm and base motion and force dependency. 
Based on the dynamics coupling and measurement method, we proposed the concept of 
dynamic balance control, the use of the proposed concept is of significance in planing the 
balance arm's motion for compensate the attitude disturbance of space base. 
Based on the dynamic balance control concept, we proposed the coordinated control 
algorithm for the mission arm and the balance arm. During the operation of mission arm, 
the balance arm can easily compensate the disturbance due to motion of the mission arm. 
The performance of the coordinated control algorithm is verified by the simulation studies. 
The simulation results showed that the proposed dynamic balance control method could be 
used practically. 
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1. Introduction 

The development of microrobots on a millimeter scale has recently received much attention. 
The environments in which these robots are supposed to operate are narrow and potentially 
complicated spaces, such as micro-factory, blood vessel, and micro-satellite. The robots must 
have omni-directional mobility, high power and high load capacity, within a scale in 
millimeters, in order to accomplish the work efficiently. 

Motion principles and actuation mechanisms that combine volume, motion of 
resolution, and the speed virtues of coarse positioning, are still the challege in the 
microrobot design. Different principles to drive microrobots have been developed. The 
Microprocessor and Interface Lab of EPEL developed a 1cm 3 car-like microrobot with 
two Smoovy 3 mm motors. Sandia National Lab developed a 4cm 3 volume and 28g 
weight microrobot for plume tracking with two Smoovy micromotors with a car-like 
steering (Byrne et al., 2002). AI lab in MIT designed Ants microrobot with a 36.75cm 3 
volume and 33g weight, driven like a tank with pedrail (Mclurkin, 1996). Caprari and 
Balmer built another car-like microrobot with 8cm 3 volume by watch motor (Caprari et 
al., 1998). Dario developed a millimeter size microrobot by a novel type of 
electromagnetic wobble micromotor, with a three-wheel structure (Dario et al., 1998). 
Besides the normal motors driven principle, other microactuation techniques based on 
smart materials have been devised, such as piezoelectric actuators, shape memory 
alloys, etc. The MINIMAN robot and the MiCRoN microrobot have employed these 
techniques (Schmoeckel & Fatikow, 2000; Brufau et al., 2005). The first walking batch 
fabricated silicon microrobot, with the 15x5 mm 2 size, able to carry loads has been 
developed and investigated. The robot consists of arrays of movable robust silicon legs 
having a length of 0.5 or 1 mm. Motion is obtained by thermal actuation of robust 
polyimide joint actuators using electrical heating (Thorbjorn et al., 1999). 
Omni-directional mobile robots have kept developing due to inherent agility benefits 
(Williams et al., 2002). The mechanisms can be divided into two approaches: special 
wheel designs and conventional wheel designs. Fujisawa et al., Ferriere and Raucent 
developed the universal wheel for omni-directional mobility (Fujisawa et al., 1997; 
Ferriere et al., 1998). Muri and Neuman developed the Mecanum wheel similar to the 
universal one (Muir & Neuman, 1987). West and Asada developed the ball wheel 
(West & Asada, 1997), while Killough and Pin developed the orthogonal wheel 
(Killough& Pin, 1994). 
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These special wheel designs have demonstrated good omni-directional mobility; however, 
they generally require complex mechanical structures. Other researchers have tried to 
develop the omni-directional vehicle by conventional wheels. Boreinstein, et al, designed the 
omni-directional structure using steered wheels (Boreinstein et al, 1996), while Wada and 
Mori used active castor wheel (Wada & Mori, 1996). Mobile microrobot and omni- 
directional mobile robot have been well developed recently (Kim et al., 2003). However, few 
omni-directional mobile microrobot have been reported. Specially-developed wheels are 
very difficult to realize on millimeters scale due to their complexity. Furthermore, these 
structures have limited load capacity with slender rollers. Conventional wheels are the 
feasible solution for omni-directional mobile microrobot within 1cm 3 volume, due to their 
inherent simple structure. However, the microactuator within 10 mm 3 with high power 
output is still a challenge at present. 

This paper aims to present such an omni-directional mobile microrobot within the volume 
of 1cm 3 for microassembly. Microassembly is one chief application for mobile microrobots. 
Most reported mobile microrobots for micro assembly are based on piezoelectricity 
actuators to meet the high requirement of position precision. However, the piezoelectricity 
actuators usually suffer from complex power units that are expensive and cumbersome and 
which do not easily allow for wireless operation. Furthermore, piezoelectric actuators are 
complex systems that exhibit non-linear behavior and as a result they lack an accurate 
mathematical model that can provide a reliable prediction of the system's behavior 
(Vartholomeos, 2006). This chapter aims to present the construction of an omni-directional 
mobile microrobot system, with the microrobot less than 1cm 3 volume and its unique dual- 
wheels driven by electromagnetic micromotors in a 2mm diameter for purpose of 
microassembly in narrow space. The design, fabrication, kinematics analysis, and control of 
microactuator s and microrobots, are to be discussed with details of the sub-areas. 

2. Design of omni-directional microrobots on a millimeter scale 

Like macrorobots, microrobots are composed of electromechanical systems, mainly chassis 
planes and wheels units. In this section, the construction of omni-directional microrobots on 
a millimeter scale, the design of novel dual-wheel structure for microrobots and axial flux 
electromagnetic micromotors for dual-wheels, and fabrication of the stator winding for 
micromotors are to be described in sub-sections. 

2.1 Structure of omni-directional microrobots 

Two generations of omni-directional mobile microrobots, OMMR-I and OMMR-II, as shown 
in Fig. 1 and Fig. 2, have currently been developed. OMMR-I, on a scale of 8mmx8mmx6mm, 
is constructed with two dual-wheels; while OMMR-II with three dual-wheels, is with scales 
in 9.8mmx9.8mmx6mm. 

The omni-directional microrobots consist of two or three novel designed dual-wheels, to be 
described in Section 2.2. These dual-wheels, connected with each other by a set of gears and 
driven by specially-designed electromagnetic micromotors, to be described in Section 2.3, are 
evenly distributed on the chassis plane. The set of gears are fabricated by LIGA (Lithographie 
GalVanoformung Abformung) process, with a gear ratio of 1:3. Each dual-wheel structure needs 
one separate micromotor to produce the translation movement, meanwhile, the rotation 
movement of all dual-wheel structures is produced by one single micromotor. 
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(a) Structure of OMMR-I (b) Photo of the OMMR-I 

Fig. 1. Structures and photos of the omni-directional microrobot-I (OMMRI). 





(a) Structure of OMMR-II (b) Photo of the OMMR-II 

Fig. 2. Structures and photos of the omni-directional microrobot-II (OMMRII). 

All translation micromotors are controlled as one motor, to rotate synchronously. The active 
gear, in the middle of the chassis, is driven by steering micromotor, and the passive gears 
are connected to dual-wheel structures through an axis perpendicular to the chassis plane. 
Power from the steering micormotor is transmitted through gears to the axis, and then to the 
dual- wheels via friction between the wheels and ground. Therefore, all dual- wheel 
structures keep the same direction at any moment. Moreover, this set of microgears can also 
amplify rotary driving power and improve the rotary positioning accuracy of microrobots. 



2.2 Design of novel duel-wheel structure 

Conventional wheels for omni-directional mobile robots can generally be divided into three 
types, centred wheels, offset wheels, and dual-wheels, as shown in Fig. 3. Mobile robots 
with centred wheels must overcome dry-friction torque when reorienting the mobile robots 
because of the fixed wheels, however, mobile robots with dual wheels, kinematically 
equivalent to centred wheels, only need to overcome rolling friction. Moreover, compared 
with robots with offset wheels of identical wheels and actuators, robots with dual-wheel 
structure can double the load-carrying ability by distributing the load equally over two 
wheels. However, the complexity of an omni-directional mobile robot with a conventional 
dual-wheel structure can not be applied into microrobots with scales in millimeters. 
Therefore, a new dual-wheel structure is required to be designed for an omni-directional 
microrobot on a millimeter scale for a microassembly system. 
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(a) centred wheel (b) offset wheel (c) dual wheel 

Fig. 3. Structure of the three types of conventional wheels. 

This novel duel-wheel structure, as shown in Fig. 4, is composed of two traditional coaxial wheels, 
separated at a distance and driven by a electromagnetic micromotor, to be described in Section 2.3. 
The characteristic of this design is that dual-wheels are driven by only one motor and by frictional 
forces independently, instead of the two motors. The goal of this design is to keep the volume of 
microrobots within 1cm 3 through simplifying the structure of micromotors; meanwhile, 
mircorobots can have omni-directional mobility, high load capacity and positioning accuracy. 
This novel dual-wheel structure has certain advantages over single-wheel designs and 
conventional dual- wheels. Single- wheel structures produce relatively high friction and 
scrubbing when the wheel is actively twisted around a vertical axis. This will cause slip motion, 
therefore, reducing the positioning accuracy and increasing the power consumption, a crucial 
parameter for a microrobot. The scrubbing problem can be reduced by using dual- wheels. 
However, in ordinary dual-wheel structures, both wheels are driven by two independent 
motors, which will increase the complexity of the construct and the size of the structure for a 
microrobot. This new structure can change the dry-friction between the dual-wheels and the 
ground into rolling resistance during its steering and keep the small volume of microrobots as 
well. Two coaxial wheels, namely, active wheel and passive wheel, are connected to one 
micromotor shaft on both sides. The active wheel is fixed to the shaft driven by the 
micromotor; meanwhile, the passive wheel has rotary freedom around the shaft, driven by 
frictional forces between itself and the ground. Friction during translation leads to the active 
wheel and the passive wheel rotating synchronously, however, the two wheels rotate in 
opposite directions during steering. Therefore, omni-directional mobility with reduced wheel 
scrubbing on a millimeter scale is produced by this dual-wheel structure design. 
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Fig. 4. Structure of novel dual-wheel. 



2.3 Design of axial flux electromagnetic micromotor 

Actuators are a crucial part in designing microrobots, mainly because of the lack of currently 
available micromotors and the unsatisfying performance of existing ones. Forces, such as 
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electrostatics, piezoelectricity hydraulics, pneumatics, and biological forces, scale well into 
the micro domain, but some of them are difficult to be built in millimeters , size. 
Electromagnetic forces can give micromotors larger output torque (Fearing, 1998) and longer 
operating lifetime than others in the same volume. Electromagnetic micromotors, such as 
smoovy micromotors and IMM (Institut fiir Mikrotechnik Mainz GmbH) micromotors, are 
designed with radius flux structure. However, the height of micromotors is several times 
larger than the diameter. Therefore, in this section, an original axial flux electromagnetic 
micromotor, as shown in Fig. 5, is designed with the following characteristics: 



Am* 

I'- ii .:i , 

Fml HmYr 

Wincliwip 

Khi Lhif nuij'jvlLi plLvir. 



Fig. 5. Structure of the 2mm micromotor. 

• the axial magnetic field shrinking the volume of micromotor 

• a novel structure consisting of one rotor set between two stators 

• the rotor having multipolar permanent magnets with high performance 

• the stators having slotless concentrated multilayer planar windings 




2.3.1 Structure and analysis 

Electromagnetic micromotors, according to directions of magnetic flux, can be divided into 
two types, radial flux and axial flux micromotors, as shown in Fig. 6. Comparing with radial 
flux micromotors, axial flux ones can improve the efficiency of electromagnetic energy 
transformation, and enlarge the electromagnetic interaction area between a rotor and a 
stator, the most important parameter for a micromotor. An electromagnetic micromotor 
with a magnetic flux 'sandwich 7 structure — two stators in outliers and one rotor inside — for 
enough torque output is designed shown in Fig. 5. 
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a. The structure of axial magnetic field b. The structure of radial magnetic field 

Fig. 6. The structure of electromagnetic micromotor. 

According to the principle of T =BILr , the torque output (T), a critical measuring parameter for 
micromotors, is directly proportional to the magnetic flux density in the gap (B), current value of 
winding (I), valid winding length (L), and spinning radius (r). Although the design of multilayer 
windings has been adapted to increase the valid winding length, the overall micro size of 
micromotors limits values of L and r. Hence, the magnetic flux intensity becomes the key factor in 
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improving the performance of the micromotor. The selection of magnetism materials with high 
properties and the design of an optimum magnetic circuit become key factors to improve torque 
output. In current research, the stator winding has been designed in slotless and multiple layers, 
therefore, the air gap in the electromagnetic micromotor can include the height of the stator 
winding itself. This results in a difference in the magnetic flux density between the winding layers. 
The relationship between the flux strength and the air gap width is shown in Fig. 7. 




6 8 10 12 

Air gap (lOjum) 

Fig. 7. Relationship between magnetic flux density and air gap. 

The attractive force, in the z direction between the rotor and the stator, increases sharply as 
the gap (g) decreases. When g is 0.05mm, the attractive force reaches 27.7mN, one thousand 
times larger than the weight of the rotor (<2mg). Therefore, the friction force caused by the 
attractive force will be much larger than that caused by the weight of the rotor. 

2.3.2 Optimal design of micromotor parameters with genetic algorithms (GA) 

2.3.2.1 Targets of the design 

Performance indices, such as efficiency, torque output, speed, and operating lifetime, can be 
used to measure a motor. Two of them are selected as main targets in this design: 

• larger torque output 

• less loss of power 

The torque output is a key index evaluating the performance of a motor. The heating loss of the 
windings is the main loss of power in micromotors. It will affect the operating lifetime of 
micromotors. Although the absolute value of this loss is not large, it is still crucial to the operating 
lifetime of electromagnetic micromotors because of the overall micro size and the high intensity of 
power. Therefore, the less loss of heating power is defined as another target of the design. 



2.3.2.1 Mathematical model of the micromotor 

Having selected the targets of this design, with the purpose of applying genetic algorithms 
(GA) into this design, the mathematical models of electromagnetic micromotors has been 
drawn as follows: 
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T is the torque output of single phase 

m is the number of the layer of the stator 

n is the number of the turn of the winding 

Bi is the magnetic flux density of the i th layer of the stator 

I is the value of rated current 

Lj is the average effective length of a coil in a phase of winding 

r is the average radius of the circle track of the centroid of effective winding 

r\ is the compromise coefficient 

Ph is the loss of heating of single phase 

R is the value of resistance of single phase winding 

p is the conductive coefficient of copper 

li is the length of a circle in single phase winding 

S is the area of the wire in winding 

b is the width of the wire in winding 

h is the height of the wire in winding 

E is the back EMF of single phase 

N is the speed of the motor ; 

The above formulas show that larger output and less heating loss are a constraint 
satisfaction problem (CSP) (Li & Zhang, 2000). Larger torque output can be obtained 
through either increasing layer numbers of the stator or loop numbers of the winding, 
however, both of them will lead to more heating loss. Meanwhile, the increase in the 
layers of the stator will result in a larger air gap, corresponding to smaller values of 
magnetic flux density. Likewise, the torque output will drop when decreasing the 
heating loss. The solutions to the constraint satisfaction problem are to be discussed in 
the following subsections. 

2.3.2.2 Application of GA in the micromotor design 
• Definition of objective function 

The application of GA in this design is put forth to solve the CPS in the design of the 
micromotor. As the only dynamic factor to guide the search of GA, the value of objective 
function, cp, directly affects the efficiency and result of algorithms. The objective function 
should combine with specific design targets for its reasonability in physics. Therefore, in this 
design of the micromotor, the power of the micromotor has been selected as a bridge to 
combine the torque output and the heating loss. Through changing coefficients, different 
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parameters can be reached to satisfy various applications of the micromotor. The objective 
function in physics can be described as follows: 



--A(TN/9550)-ju(l 2 RJ 



(6) 



Because of the low efficiency of micromotors, the objective function will not keep 
positive value in its domain, which will result in the low efficiency of the algorithm. 
Hence, in this research, it is not suitable for the search to value the objective function 
from the limitation of GA. A basic positive constant is required to be added to make 
the signature of (p positive during the search without changing the physical meaning of 
(p. Another issue, to be considered, is that the number of design variables is not unique. 
The large domain of each variable, leading to the large domain of the objective 
function, will bring negative effect to the search of GA. Therefore, the space of 
objective function has been compressed by using the mathematical method, logarithm, 
keeping the signification of objective function. 



q> = ln(BASE + A{TN/9550) + ju{i 2 r}) 



(7) 



• Definition of variables 

From the formulas (1) and (2), it can be seen that the constraint satisfaction problem (CSP) 
between torque output and heating loss is embodied in parameter contradictions of the 
layer number, the circle number and the height of the stator winding. As a result, the three 
parameters are defined as the variables in the design of the micromotor. 




•a ,: 



















<#«■■■ ■*■ i>iiiii 



■ 



*'|" "»•»»** •l*"****""^* 1 



29 » 3S3 



Jut 



W J» 3D H3 



Fig. 8. The curves expressing the application of GA in the design of micro-motor. 
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The search space of each variable is defined by combining its physical meaning and the 
constraints fo micromanufacture as following: 

6 - [m,n,h\ 

me [1,8]; ne [1 , 14] ; he ( 0.0 , 16.0 ) 

The search state space is shown in the first chart of Fig. 8. Other charts in Fig. 8 show the 

average value of the fitness, the best value of the fitness, and the departure value of the 

fitness of each generation, respectively. 

The best solution obtained by the application of GA is shown as following: 

m=4, n=9, H=14.24mm 

2.4 Fabrication process of the stator winding 

According to the results of optimal design with GA, a stator winding of the electromagnetic 
micromotor in a diameter of 2mm is manufactured by microfabrication technique. The stator 
winding, composed of 4 layers of coils, 54 turns, and 3 pairs, is only 2mm in diameter and 
lum in minimum line space with the maximum operation current of the coils of 230mA and 
resistor is 22-30Q. The structure of the winding is shown as Fig. 9. 
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(a) Top view Figure (b) Section Figure 

Fig. 9. Structure of the structure. 

The substrate is a special ferrite wafer, 4mm in thickness and 3-inch-diameter. In total 13 
masks are required during the winding process of the stator, and coils and connectors are 
fabricated by mask-plating process (Zhao et al. 1999). 4 layers of alumina isolation layers are 
deposited by a sputter machine. The basic processes of the stator winding are described as 
follows, and the flow charts are shown in Fig. 10. 

1. Depositing a seed-layer (Cu/Cr or Cu/Ti) with lOOnm thickness on the ferrite 
substrate for the electro-plating by sputter process. Cu is the electrode for 
electroplating, while the Cr or Ti is used to increase the adherence force between the 
seed-layer and substrate. The processes are shown in Fig. 10(a, b). 

2. Spin-coating a photoresist layer, which is patterned to form the mask for electro-plating 
the windings after the exposure and developing processes, shown in Fig. 10 (c, d) 

3. Electro-plating from the seed layer to form the windings with designed structure, 
shown in Fig. 10 (e) 

4. Spin-coating the second photoresist layer, which is patterned to form the mask for 
electro-plating the connecting wire between the two adjacent windings layers, after 
the exposure and developing processes, shown in Fig. 10 (f, g) 
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5. Electro-plating to form the connectors between the two adjacent layers windings, 
shown in Fig. 10 (h) 

6. Removing the photoresist layer by actone, shown in Fig. 10 (i) 

7. Removing the seed-layer by sputter etching process, shown in Fig. 10 (i) 

8. Depositing an insulation layer (alumina) by sputter process, shown in Fig. 10 (j). 

9. Removing the unwanted part of the insulation to bare the connectors between the 
two layers by lapping and polishing, shown in Fig. 10 (k) 

10. Fabricating the second winding layer by repeat the above steps. 

The fabrication difficulty of the stator increases with the increasing of the number of 
winding layers. Structure design of the micromotors for microrobot of this project select a 4- 
layer structure according to the results of the optimal design with GA. The structure of the 
stator is shown in Fig. 11. 
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Fig. 10. Fabrication process of the stator winding. 
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Fig. 11. Photo of the stator. 




Fig. 12. Photo of the micromotor and a sesame seed. 



The rotor is made from SmCo permanent magnetic alloy. A special magnetization method 
has been developed to write pairs of magnetic poles on the surface of rotor in the vertical 
direction. The photo of the micromotor in contrast to the sesame is shown in Fig. 12, and its 
performance is shown in Table 1. 



Over size 
(mm) 


Max speed 
(rpm) 


Weight 

( m s) 


Range of 
timing 


Max torque output 
(uNm) 


2.1x2.1x1.3 


28000 


38 


50:1 


2.8 



Table 1. Main performance parameter of the micromotor. 



3. Kinematics characteristics of the omni-directional microrobot 

The structure of the novel duel- wheels has been described in previous section of this chapter. 
The active wheel is fixed to the micromotor shaft, while the passive wheel has the rotary 
freedom around the shaft. When the duel-wheel turns, the vertical shaft (transmission gear 
shaft) doesn't move. Therefore we can simplify the complicated wheels as a single virtual 
wheel locating in the center of the duel-wheel with zero thickness, which is drawn as the 
broken line in Fig. 13. In the coordinate systems defined in Fig. 13, 
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Fig. 13. Coordinate systems of a dual-wheel. 

C r = (O r , X r , Y r , Z r ) is the robot inertial coordinate frame 
C = (O , X , Y , Z ) is the ground inertial coordinate frame 
y/ the angle which C r offset C 

x a k x Coordinate of the point the Kth active wheel contacting with ground 
y ak y Coordinate of the point the Kth active wheel contacting with ground 
x pk x Coordinate of the point the Kth passive wheel contacting with ground 
y pk y Coordinate of the point the Kth passive wheel contacting with ground 

x k x Coordinate of the point the Kth virtual wheel contacting with ground 

y k y Coordinate of the point the Kth virtual wheel contacting with ground 

k (t) Rotation angle of the Kth virtual wheel from the vertical axis at time T 

(/) k (t) Rotation angle of the first virtual wheel from the horizontal axis at time T 

Since the microrobot moves on a plane, the coordinate C r is chosen to satisfy the analytical 

requirement. Therefore, the pose vector £ consists of the Descartes coordinate of the 

reference point O r in coordinate Co and the offset angle C r from Co. Because the microrobot 

carried by the duel-wheels moves on a plane, its motion has three degrees of freedom. 

Where the microrobot consists of n (n=2, 3) duel-wheel structures, the microrobot state (s) 

can be expressed with 3+2xn vectors (Alexander, 1989): 

Where, 

e=[9i e 2 ...e n ]T 

and 

4 >= [4 > i ^2 ...<M T . 



3.1. Kinematics constraints of the microrobot wheel 

While the microrobot moves, the wheels only roll on the plane without slip. The velocity of 
the contact point between the virtual wheel and the ground is zero. 



-x-sin(y/+0 k ) + y-cos(y/+0 k ) + (x k - cos k +y k -sm0 k )i/s = O 



(8) 
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x-cos(y/+0 k ) + y-sm(y/+0 k ) + (x k -sm0 k -y k -cos0 k )ys-r k -(f> k =O 



(9) 



Therefore, the steering motion of the microrobot and the angular velocity of the virtual 
wheel can be drawn as following: 



k = Arctg 



- x • smy/ + y • cosy/ + x k y/ 
x-cosy/+y-s\ny/-y k y/ 



. _x-cos(y/ + k ) + y-sin(y/ + k ) (x k -sm0 k -y k -cos0 k )iy 



(10) 
(11) 



The microrobot kinematics constraints can be expressed by considering the n duel-wheels. 



where 



4o)- 



m= 
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Assuming A(s) expresses the constraints matrix of microrobot kinematics, 

A{s) (K(e)-R{ ¥ ) 
[jM-rM o -j 2 



then the kinematics constraint equation is 



(12) 
(13) 

(14) 
(15) 



A(s)s=0 



(16) 



(17) 



(18) 



3.2. Kinematics analysis of the omni-direction microrobot 

The kinematics of the omni-directional microrobot is used to analyze the possible mobility 

under the kinematics constraint equation (11). 

1) The Mobility 

From the equation (12), the state vector R f (y/)-i belongs to the zero space of matrix K (#) . This 

produces the movement constraint of the microrobot, while the equation (13) doesn't constrain 
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the movement The condition can be satisfied by giving the vector a suitable value: 

i = J- l -A(§)-R<yY£ ( 19 ) 

Therefore, the zero space of is only to be considered to study the mobility of microrobot. k(&) 

When the turning angle the microrobot is fixed, the matrix is nx3 because the microrobot 
has n duel- wheels. When n > 3 , the matrix order is 3 normally, which means the robot has 
good mobility on a plane. However, as n=2, the matrix order is 2 normally, which means its 
order is equal to 1 only when the head direction of the microrobot is perpendicular to the 
connecting line of the two castor centre points. Therefore, the velocity vector of the 
microrobot is a one-dimension space except that the special situation £ is a two-dimension 
space. This is solved by adding two supporting points in the design. Thus, the microrobot is 
proved to have only one freedom when it moves around the plane. 
2) The Directionality 

Under the constraint of equation (12), the directionality is defined to the microrobot 
movement when the vector changes with time. 

The angle of the microrobot to the frame , can be obtained by the following equation, 
while the state of the microrobot and the velocity vector at this moment are known: 

K{e)-R t { ¥ )-v_ = (20) 

It proves that the microrobot can achieve omni-directional mobility under the kinematics 
constraint of equation (12). 

4. Control of the omni-directional microrobot 

The 2mm micromotor, with the 8-polar rotor and the 9-coil stator, has been designed as a 3- 
phase synchronous motor in the star-connected windings. Controlled with 2-2 phases 
conduct approach (2-2 PCA) and 3-3 phases conduct approach (3-3 PCA), micromotors can 
work as a synchronous motor with different speeds (reference 10). 2-2 PCA means two 
phases conduct current at any time and leave the third floating, whose vector figure is 
shown in Fig. 14(a). AB represents that current in windings is from A to B via the middle 
point. 3-3 PCA means all the three phases are conducted. MC represents that current in 
windings is from both A and C to B via the middle point. In order to produce maximum 
torque, the inverter must be commutated every 60° electrical angle. Therefore the 
micromotor need change 6 steps rotating 360° in an electrical angle, while 24 steps in 
mechanical 360°, shown in Fig. 14(b). Micromotors can be used as stepper motors with these 
two approaches. However, the accuracy is not high enough for the mission in micro fields, 
such as micromanipulation. 2-3 phases conducted approach (2-3 PCA), whose vector figure 
is shown in Fig. 14(c), can only increase the step accuracy of micromotors by two, which still 
limit applications of micromotors. Therefore, two novel approaches, Virtual- Winding 
Approach (VWA) and PWM (pulse width modulation) Based Vector-Synthesize Approach 
(PBVSA) are designed to improve the output torque and positional accuracy of 
electromagnetic synchronous micromotors without changing their structure. Both 
approaches are designed to control the value and direction of the current in each phase, then 
to change value and direction of synthetical magnetic, therefore to increase the steps of 
micromotor in 360° by electrical angle. 
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4.1 Virtual-Winding approach (VWA) 

The VWA is realized by connecting the central point of the phases with a virtual winding, denoted 
as R', controlled by PWM signal, shown in Fig. 15. Through changing the frequency of the PWM 
pulse, the average value of the virtual winding is selected to change the current value of each phase. 
In VWA step control, the current through phase A, B, C and R' are denoted respectively as 
il, i2, i3 and i', shown in Fig. 15. Where the R' is inserted into the circuit conducted as AB^y 
parallel connection with phase A, expressed asxAB. x represents the pulse duty cycle of the 
control PWM. The torque vector is shown in Fig. 16. If the steps required are fixed, thenX is 
a fixed value which can be computed, and the torque value will also be a fixed value. 
In this approach, 72 steps in 360° by mechanical angle are realized by inserting two steps 
into one step in 2-2 PCA. Then the communication phases sequence is: 

AC + xAC + xBC + BC + RxC + xAB + AB + AxB + AxC + AC + xAC + xBC + 
BC+lkC+xAB+AB + JxB+AxC+AC 

Compared with 2-2 PCA, there are 2 new steps with the VWA in a control step, which 
means 60° in electrical angle^ shown in Fig. 15. According to law of sines, the expression can 
be drawn from Fig. 16. 

/2_ sin50° _l + x 

il sin 10° x (21) 

Therefore, 

x = 0.293 (22) 




^.ABC AB 9-' 



/' ABC ABC"-. 
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(a) 2-2 PCA control (b) 3-3 PCA control (c) 2-3 PCA control 

Fig. 14. Vectors in an electrical angles 360°. 




13 n 

Fig. 15. Virtual winding principle for current divided. 
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Fig. 16. Virtual winding control for step precision increasing. 

Fig. 17 shows the experiment waves with the micromotor under 2-2 PC A control and virtual 
winding control. The black bumps in Fig. 17(b) are the voltage wave of the inserted steps. In 
theory, higher accuracy can be achieved in this approach by adding one extra PWM signal 
to the virtual winding. However, the virtual winding will reduce the output torque, 
incapable of providing constant torque output. This will weaken the performance of 
micromotors and applications will be limited as well. 





(a) 2-2 PCA control (b) Virtual coil control 

Fig. 17. Voltage waveform of micromotor under 2-2 PCA control and VWA control. 



4.2 PWM-Based-Vector-Synthesize approach (PBVSA) 

PBVSA produces microstepping with a constant torque output for a micromotor, by 
utilizing the third coil in the three star-connected windings to divide currents into two parts, 
to synthesize the magnetic field, and to insert required steps. 

PBVSA is based on the theory of torque vector synthesis. In 2-3 PCA control, when the stator 
is conducted with the current of AC at this time, the rotor will stop at its position. Then the 
stator will be conducted with the current of ABC* and the rotor will stop at a position with a 
30°deviation in electrical angle. When two circuits are conducted alternately by two PWM 
signals, with one high frequency, the synthesis torque can be controlled by changing the 
ratio of these two currents, the stop position of the rotor is then decided. The frequency of 
PWM signal is at least ten times as much as that of commutation. Where the period of the 
PWM1 and PWM2 signals are t, tO respectively, and tl represents the conducted time of the 
two signals, the current through winding A (Ia) can be divided into two parts Ia2 and Ia3/ 
representing the currents when the stator is conducted as AC and ABC, respectively. As such 
the current IC can be divided into Ic2 and Ic3. Their time sequence has been shown in Fig. 18. 
ince the two circuits cannot be conducted simultaneously, then: 
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Fig. 18. Time sequence between PWM signals and every phase's current. 



t>t +t l 



(23) 



Assuming the resistance of each phase winding is R, the stator is conducted with a voltage 
of U, and the average value of Ic3 in a period is: 



2 U S n 

U — - + 
. 3R 



(24) 



Assuming p is pulse duty cycle of PWM2, and p=ti/t. I is the rated current, / 
Similarly, the average value of Ic2 in a period is: 



2U S 
3R 



2R 



- + 



3/ 



(25) 



Assuming a is pulse duty cycle of PWM1, and oc=t0/t. 
Therefore the average value of Ia3 in a period is: 



. 3R 



and the average value of U2 in a period is: 



I a?. - Icr - a ~ 



3/ 



while the average value of Ib is : 



3R 



-Pz 



(26) 
(27) 

(28) 



Since the magnetic field of the 2mm micromotor is designed with a trapezoid shape 
distributed in the gap between the stator and rotor, the average torque produced by Ia2 and 
Ic2 can be expressed as: 
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T 2 =BLSa^ r - = c(r (29) 

2 4 2 

Where: B is the intensity in magnetic field between the rotor and stator; L is the valid length 
of each phase winding; r the average radius of the windings; T ^ BL J^H^Lis the largest 

2R 2 

value of the torque output under 2-2 PC A control. Under 3-3 PC A control, the average 
torque produced by Ia3% Ib3 and Ic3 is: 

F 3 =^V (30) 

The vector relationship among T M , T 2 and T 3 must be kept as shown in Fig. 19. According 
to the law of cosines, 



AC(T) 




T- 

Fig. 19. Vector relation among T~ M , f\ and j^ while the synthesis torque is a constant. 



„2 +( ^2_ 2a ^ coi( 5 i r ) (31) 



3 3 6 

Therefore, the synthesis torque output is not decided by the required steps, which means a 
constant torque output can be obtained by PBVSA. 

To get Tm with the constant value of T, the following constraints must be obtained according 
to law of sine: 

sin(— -0) 

a = £- = 2sin(--<9) (32) 

• 5ft 6 

sin — 

6 

V^m^ = ^ sm6? (33) 

2.5^- 
sm — 
6 

Thus, the microstepping with constant torque output for an electromagnetic micromotor can 

be obtained by changing the values of a and p. 

Microstepping with constant torque for a star-connected PM micromotor is realized 

without any additional circuit to change the value, which will result in a higher efficiency 

compared with VWA. However, this method requires a more complicated control circuit 

to produce the time sequences. At present, experiments of 144 steps in 360° by mechanical 

angle have been obtained under PBVSA control. The control waves, the phase voltage 

changing relatively to the ground, and the amplified part, are shown in Fig. 20(a) and Fig. 
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(b) Amplified waves 
Fig. 20. Waves of voltage between coil conducted and ground under 3-3 PCA control and 
PBVSA control. 

The statistical data of the deviation angle, obtained from the real position and the designed 
angle during a 360 mechanical angle in the experiment, is shown as Table 2. The deviation is 
led by the uneven magnetic field intensity at the edges of the magnetic poles. 
Both Virtual-Winding Approach (VWA) and PWM Based Vector-Synthesize Approach (PBVSA) 
are designed to realize the microstepping of microrobots without changing the present structure of 
micromotors. These are limited by the present microfabrication process, and to explore the new 
applications for electromagnetic micromotors. Though these novel approaches are demonstrated 
by the 2mm micromotor, they can be applied for the electromagnetic micromotors with similar 
structures like PM motors, with either axial or radial direction magnetic field. 
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Deviation angle 


0-0.2° 


0.2-0.3° 


0.3-0.5° 


Number of steps 


75 


36 


33 



Table 2. Test result of the micromotor under PBVSA control. 



5. Application of the omni-directional microrobot 

The omni-directional microrobots within 1cm 3 (shown in Fig. 21), were designed for 
microassembly in narrow space, and have been applied into a microassembly system, to 
manipulate and transport a microbearing with a 0.6mm inner diameter and a 1mm outer 
diameter, and to assemble it in an axis with 0.4mm diameter. The system is composed of 
omni-directional microrobots with a microgripper, an assembly platform, a visual system 
and a microrobots controller (shown in Fig. 22). The assembly platform is divided into an 
assembly part and a transport part. The visual system consists of three CCD cameras, a 
visual grabber card, and a computer. One normal CCD camera covers the whole platform as 
the global visual sensor, while the other microscope CCD cameras cover the assembly parts 
as the local sensor. The acquisition and tracking algorithm is run to acquire the positions 
and postures of the microrobots. The controller decides the microrobots behaviour by 
communicating with the visual system to execute the assembly mission. 





•^T^TI 
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Fig. 21. Photo of the microrobot with micro gripper. 




Fig. 22. Photo of the micro assembly system. 
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The microrobot is capable of moving continuously or in stages with a high precision. Both of 
the two novel approaches, VWA and PBVBA, are applied into this microrobot. The results 
are showed in the following Table 3. 





VWA 


PBVSA 


Translation precision 


O.llmm/Step 


0.04mm/ Step 


Rotation precision 


1.25°/Step 


0.625°/Step 



Table 3. Contrast results between two methods applied on a microrobot. 

Table 3 shows that PBVSA can achieve higher precision than VWA under the same 
controlling circumstances. It is because in the VWA approach, a resistor is used and thereby 
consumes part of the power. However, its simpler circuit make VWA easy to produce. 

6. Conclusion 

This chapter has presented the construction of an omni-directional mobile microrobot 
system, with the microrobot less than one cubic centimeter in volume and its unique dual- 
wheels driven by electromagnetic micromotors, two millimeters in diameter, for purpose of 
microassembly in narrow space. The results show that the microrobot can move 
continuously or in stages with high precision under the two novel micro step control 
approaches, which is necessary for the microassembly mission. This omni-directional 
mobility and the high precision movement show that the omni-directional mobile 
microrobot will have many potential applications in micro fields. 

Although this work shows a good performance of a microrobot, there is still some further 
work needed for this microrobot as the precision of the microbot is not enough for the nano- 
manipulation and the power consumption of the microrobot is not very satisfying. In order 
to accomplish more complex work, the multiple microrobots cooperation control still needs 
to be adopted. 
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1. Introduction 

The research and development of various types of robots is actively being pursued, 
especially in Japan. It is expected that the Aichi EXPO robot shows and robot exhibi- 
tions would be one of the most popular shows. There are several reasons why there is a 
big boom for robots right now. One of these is that robots have physical bodies and 
because of this, communications between robots and human stretch beyond the com- 
munications between computer characters and humans. Since the capabilities of robots 
to date are insufficient for supporting us in various aspects of our lives, however, one 
of their major applications is entertainment. 

The role of entertainment in our daily lives is a very important one. Entertainment re- 
laxes us, pleases us, and thus contributes to our mental health. Although there are vari- 
ous types of entertainment, the positioning of dance is unique as dancing activates both 
our body and our brain. Furthermore, dance can be considered a kind of communication. 
It is well known that there are two types communication: verbal and nonverbal, and 
dance is one of the most sophisticated nonverbal communication methods. 
If we could develop a robot that can dance, it would be very interesting from various 
points of view. First, the dancing robot might become a new form of entertainment. To 
watch humans dancing is one of the well-established forms of entertainment, but if we 
could show people various types of dance achieved by robots, that would be a new type 
of entertainment. Second, we would be able to develop a new type of communication 
with computers. The interactions/ communications between humans and computers 
have long been studied, with most of the research treating communications using 
speech, facial expression, and gestures commonly used in our daily communications. 
Although these are important research themes, it is necessary for us to tackle the prob- 
lem from a different aspect. Because dance is a special type of communication, by study- 
ing a dance robot, it is possible that we could develop a new type of communication 
between humans and robots. 

Based on the above considerations we have commenced research on robots that can dance. 
In this paper, we clarify the relationship among entertainment, humans, and robots. We also 
describe an example of a robot that can dance. 
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2. Entertainment and Robot 

2.1 Entertainment 

There are so many aspects to consider and discuss concerning entertainment (Callois, 
1958)(Csikszentmihalyi,1975). One of the most important aspects may be the existence 
of sides: the side to entertain and the side to be entertained. Although these two sides 
might exchange their positions with one another depending on the situation, the exis- 
tence of performers and spectators is an absolute condition for entertainment. Many 
types of entertainment have characteristics whereby people can become both enter- 
tainers and spectators. In the case of baseball, for example, people can play baseball 
and thus entertain themselves, and they may also enjoy it by watching professional 
baseball games as spectators. Dance belongs to this category. People sometimes at- 
tend theaters to watch good dance performances and may sometimes go to dance 
clubs or discos to enjoy dancing. 

Furthermore, entertainment could be classified into two categories when looking it from a 
different aspect. One is a real-time type, where performers or entertainers are in face-to-face 
mode with spectators. Good examples are plays and/ or concerts at a theater. The other is 
the non-real-time type. This includes reading books and watching movies. The advantages 
and disadvantages of each type are listed below. 
Real time type: 
Advantages 

-A performer can observe and feel a spectator's reaction. 

-A spectator can enjoy the feeling of presence. 
Disadvantage 

-When a performer makes a mistake, it is difficult to redo the action. 
Non real-time type: 
Advantages 

-Prior editing of the performance is possible so that a perfect performance can be shown. 

-It can be repeated any number of times, at any time. 

-Prolonged preservation is possible. 
Disadvantage 

-It is difficult for spectators to have the feeling of presence. 
Following this classification, dance basically belongs to real-time entertainment. In the 
case of robot dancing, however, as will be described later, its positioning is somewhat 
special. 

2.2 Positioning of entertainment robot 

In recent years the robot has become a center of attention (Levy, 2005). The dog- type robot 
called AIBO has attracted a lot of people (Golubovic et al., 2003)(Ishimura et al., 
2003)(Kerepesi et al., 2006), especially women and aged people who wanted to have a pet. 
Also, a walking robot called ASIMO by Honda surprised many people by its walking dem- 
onstration (Ishimura et al., 2003). In addition to these pioneers, currently various types of 
humanoid and pet robots are being studied and developed in many universities, company 
laboratories, and venture companies. Furthermore, various demonstrations, shows, and 
exhibitions are being held, with performances as a humanoid robot's offering relief, dancing, 
fighting, among others (Nakaoka, 2003) (Shira tori, 2004). 
However, there is still the possibility that these are temporary phenomena and sooner or 
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later the boom will subside. Although it is expected that in the future these robots will help 
us in various aspects of our daily lives, so far their capabilities are very limited. For example, 
we expect that at homes they will assist aged people by helping them to walk, bathe, and so 
on. At present, however, it is sill very difficult to install one of these tasks onto a robot, one 
reason being that robots cannot behave intelligently, making it difficult at best for us to rely 
on them. Another problem is that it is necessary for them to have high physical strength yet 
at the same time delicate behaviors if we want them to really be useful. For these reasons 
still there is only a small market for humanoid/pet robots. 

To break through this barrier, entertainment is expected to be a good application area for 
robots (Nakatsu & Hoshino, 2003). With the above-mentioned entertainment robot there 
are various ways for users to enjoy themselves, such as creating, operating, showing, and 
watching. In the case of entertainment, it is not necessary for a robot to have a strong 
physical power; what it should do is to entertain us by its behaviors and interactions with 
us. Thus it would be appropriate for us to focus on the application of robots to entertain- 
ment. Although there are various types of entertainment, here we focus on dance. 
One main reason we choose dance as entertainment by robots is that dance is one of 
the most sophisticated forms of entertainment. Based on the considerations described 
in Sections 3.1, what would be the role of robots in dance entertainment? One signifi- 
cant point is that a dancing robot would make it possible for us to become both enter- 
tainers and spectators. To the extent that we watch a robot dancing, we are spectators; 
on the other hand, it is expected that many people would want to install dance motions 
onto their robots and would want to show their robot motions to other people. In this 
case they are entertainers. 

In distinguishing between real-time entertainment and non-real-time entertainment, 
again dance robots feature significant characteristics. Basically, if we want to show 
people the robot dancing, we have to install the dance actions beforehand, which 
means that the robot dance is non-real-time entertainment. Meanwhile, by developing 
interactive capabilities, the robot would perform impromptu dancing behaviors. For 
example, if the robot could recognize a request from one of the spectators, it could 
change the dance depending on the request. Or if it could sense the mood/ atmosphere 
of the spectators, it could adapt its dancing behaviors depending on the sensing results. 
This means that the dance robot could give us a sort of variable entertainment that 
ranges between real-time and non-real-time entertainment. 

3. Humanoid Robot and Its Dance Entertainment 

3.1 The function and specification of a humanoid robot 

There are already several humanoid robots available on the market. Among them we have 
selected a humanoid robot developed by Nirvana Technology, on which we will try to in- 
stall dance motions. Figure 1 shows the appearance of the robot. The moving range for each 
servomotor used for the robot is 180°. The range of movement assigned to each servomotor 
is shown in Fig. 2 and Table 1. 

The x, y, and z axes are set to the erect state, and rotation of the circumference of each axis is 
considered as roll, pitch, and yaw. Each positive and negative rotation direction is set as plus 
and minus by the right screw law. The movement range for each joint is shown in Table 1. 
The various motions the robot can perform can be designed and produced on a PC using a 
motion editor realized by software dedicated to motion creation and editing. Figure 3 shows 
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the motion editor's interface. 




Fig. 1. An example of a humanoid robot. 



S=S 




Fig. 2. The movable range of the servomotors. 



3.2 Examples of performance by robot 

As preparatory exercises for the design and development of dance motions, we have created 
several specific motions. Figure 4 illustrates Tai-chi motions we have created for the robot. In 
addition, Figs. 5 and 6 respectively display some of the robot actions dedicated to a cheering ac- 
tion, Japanese traditional dance action, and an action imitating a famous Japanese singer. 
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Table. 1. The moving range of the robot. 



Fig. 3. Motion Editor. 
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Fig. 4. Form of Tai-Chi. 
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Fig. 5. Japanese Dance. 




Fig. 6. Japanese Cheer boy. 



4. Realization of the Dance Performance by the Robot 

4.1 Definition of dance 

It is difficult to define a dance. As already indicated, there is the side to entertain and the side to 
enjoy entertainment, and dance, especially, contains both of these features. From the perspective of 
a dancer it is to entertain, and from the side of a spectator, it is to watch and enjoy. These features 
mean that dance could develop itself into a communication medium with the external world. 
Moreover, dance has the characteristic of being a pleasure to create, perform, and watch. Each of 
these factors leads to improvisational expression, making a work and appreciating it (Laban, 1980). 
Improvisational expression is not only the process of creating a work; it also involves the joy 
of the expression itself. It is based on the momentary dance act and is full of energy, indi- 
viduality and freshness that are not present in the elaborated work. 

Making a work contains the pleasure of adhering to the content that should be expressed, 
repeating a device and improving it gradually. 

Appreciation means that completion of a work by an entertainer is interpreted, with the 
spectator having the same feelings and sympathies as the entertainer. 
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4.2 The subject of dance operation by the robot 

One important issue for the dancing robot is whether it can dance in accordance with a 

rhythm. Generally, the tempo of music that can fit a rhythm is about 140 BPM ( beats per 

minute ) , and the beats are usually arranged as sets of four beats in many cases. A rhythm 
of 140 BPM has 140 standard notes in one minute; therefore, one beat lasts about 0.4 seconds. 
The software that can control the shift between pauses in the shortest period of 0.1 seconds 
can create the motion, thus the number of pauses that can be added within one beat is at 
most four. A robot's dance must fit a rhythm within this constraint. 

Another issue is whether the robot can actually attain a level of movement that can be called 
a dance. A dance uses body expressions as nonverbal communication and transmits them to 
spectators, so the message conveyed by the dance would vary like body language or a work 
of art, with originality and creativity. Therefore, understanding and misunderstanding 
might exist together between the senders and the receivers of dance information. Conse- 
quently, smooth transfer of the contents may sometimes become difficult at the expression 
stage. To alleviate the problem, a device is required to express dance that can create sympa- 
thy within the mind of any spectator. 

4.3 The generation method of dance motion 

One author of this work, Mr. Iwatani, works as a dancer at Universal Studios Japan. As we 
thought it essential to include a dancer among our members, we asked him join the group 
and start a collaboration. Based on discussions with him, rocking and popping motions 
were chosen as robot's main dances. 

Popping is a term for the dance style that makes the body flip by putting power into the 
body momentarily and combining this power with pantomime, animation, etc. according to 
a rhythm. 

Rocking is a term for a dynamic dance style with a rhythm picking of 16 beats, the combina- 
tion of lock, stops a motion momentarily and determines a pause, pointing by the fingertip 
and step etc. There is a fundamental motion but the style changes delicately depending on 
the person and place. 

Both dances consist of various segments of moves generated in accordance with rhythms. 
The sophisticated selection and concatenation of each move makes it possible for the dance 
action to be recognized as art. Furthermore, each move is generated by connecting various 
shorter segments called poses. These poses and moves respectively correspond to key 
frames and motions of the robot. This basic concept simplifies the transfer process of the 
human dancing image into robot motions. 

Several examples of the dance actions performed by the dancer are shown in Fig. 7. Also 
several examples of the robot dance action segments we have developed are shown in Fig. 8. 

5. Conclusion and Further Research 

In this paper, we have proposed dancing as a new application area for humanoid robots. First, 
we investigated the meaning of entertainment and clarified several distinctive characteristics of 
it. Then we examined the role of robots in the area of entertainment. By showing that dance 
performed by robots have several significant characteristics, we indicated that robot dance 
could become a new type of entertainment. Following that, we described several features re- 
quired for a dance robot to be a success. As an example of a humanoid robot that can be used 
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to apply dance actions, we have chosen a humanoid robot built by Nirvana Technology. 




Fig. 7. An example of the actual dance by the human. 
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Fig. 8. An example of the actual dance by the robot. 
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The most delicate point in our research is to decide and select appropriate short dance actions 
called dance segments. By merging and concatenating various kinds of dance segments it would 
be possible for our robot to perform many different dance actions. To achieve this we have started 
a collaborative study with a dancer, and based on discussions with him we are now in the process 
of collecting dance motions into a group of dance segments 

There are many research items remaining for further study. One is the evaluation of dance actions 
performed by the robot. In our laboratory, we have started measuring human brain activity by 
applying a system called MRS. The same experimental methodology could be applied for the 
case of robot dancing. We plan to measure our brain activity while watching the robot dance. In 
addition, we want to compare these data with other data such as measurement results while 
playing video games, gambling, reading books, or watching TV and movies. 
Furthermore, we want to examine whether robot dancing could be applied for image train- 
ing. It is well known that imagination includes the same function as real body training. If we 
imagine a body movement in our brain, the signal is the same as the one in the case of a real 
body action command transmitted to muscles through neurons. If such training is repeated, 
the neural network corresponding to the action will be strengthened, thus a command for 
the action can be more efficiently processed. This shows that watching the robot dancing 
and imagining those dance movements may be useful in activating our bodies. 
It has also been pointed out that watching sophisticated dance motions can relax us and release 
us from fear or sadness. Utilizing this feature, dance therapy, which incorporates dance move- 
ments for psychotherapy, has been conducted since the 1940s and remains popular. If we could 
apply robot dancing to the dance therapy, robots will have a brighter future. 
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1. Introduction 



Current research linking musical composition and computer science has traditionally 
focused on the performance through virtual musical instruments, such as within the area of 
electronic music synthesis using modern digital signal processing techniques. Synthesized 
sound, however, achieves only limited accuracy and expressive translation from actual 
performance gestures due to errors produced when, for example, converting violin string 
vibration into MIDI pitch and velocity data [1]. 

This paper focuses on the research and development of " Robot Musicians' 7 , who are 
typically referred to as Partially Artificial Musicians (P. A.M.). They perform on real 
instruments through the usage of mechanical devices. The " Robot Musician" approach is 
technically challenging and musically expressive. It opens up new areas of research and 
development endeavors. 

2. Motivation: Musical Expressiveness 

Music scores performed by robots on real instrument offer the audience live- 
experience very similar to listening to a human musician. The cognitive foundations 
and realism within real instrument performance, such as the physical vibration of a 
violin string, provides a much stronger case in music expressiveness, versus electronic 
music synthesizers. 

By controlling the parameters involved in music expressions through computer- 
controlled/ programmed mechanical entities, robot musicians are designed to bypass 
several technical difficulties that are typically encountered by human musicians, such as 
intonation, repetitive gestures, and complex articulation patterns, as well as speed and 
clarity of musical events. A Robotic Musician, potentially, could have more degrees of 
freedom in real-time performances and reach a much higher level of performance 
difficulty, flexibility and quality in terms of specific and idiomatic demands. As an 
example, one can imagine a violin being played by a robot musician with hands that have 
12 fingers. 

3. Robot Musicians Architecture 

3.1 Robot Musicians Band Overview 

A robot musician band, the P.A.M Band, has been established in the Bubble Theatre of the 
Arnold Bernhardt Center at the University of Bridgeport. Each member of the band is a 
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robot musician, which specializes in either a string or a percussion instrument. Figure 1 
depicts the P.A.M Band. Table 1 depicts the musicians 7 specialty. Figure 2 and 3 show two of 
the musicians. 




Fig.l. Robot musicians, "the P.A.M. band", designed and built by Kurt Coble. 



Robot Musician Name 


Instrument Played 


Micky 


Drum set 


Austin 


Percussion Ensemble 


Dusty II 


Electric guitar 


Jasche 


2-bow violin 


Drack 


Bass guitar 


John 


Folk guitar 


Stu 


Classical guitar 


Zak 


White electric guitar 


Dusty 


Red electric guitar 


Gold member 


Gold electric guitar 


Bernie Bot 


Cello 


Silver 


1-bow violin 



Table 1. Robot Musicians "P.A.M Band" Member Profile. 




Fig. 2. Austin plays a Percussion Ensemble. 
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Fig. 3. Dusty plays a red electric guitar. 




3.2 Robot Musician Architecture Overview 

Each robot musician adopts a three-module architecture, which consists of the following 
vertically arranged functional elements: a software module, a control module and a motion 
module. This type of modular architecture can also be found in [2]. Figure 4 shows the 
musicians 7 architecture. First, the software module interacts with users, provides the 
programming and composition environment and sends motion commands to the control 
module through a serial port. In the next step, the control module involves rules that govern 
application processing and connects the software module with the motion module. The 
control module is optimized and synchronized with the fast and repeatedly-used equation- 
solving routines. Finally, the motion module - the hands of the robot musician, is provided 
by powered mechanics, namely, servos and solenoids, which manage and provide access to 
the actual musical instrument. 
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Fig. 4: Robot Musician Architecture Overview. 

3.3 Software Module Infrastructure 

The software module is implemented as a set of Turbo C++ programs running on the DOS 
operating system. DOS has provided a robust environment for delivering continuous 
control commands to the robot musician. The software communicates with the control 
module through serial port I/O. At the current stage of development, the software consists 
of two major components: a composition component and a performance component. 
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Fig. 5. Two screen shots of Moth.C and its executing result, a C++ program which 
enables the Bubble Band to play the musical piece "The Moth". The moth can be 
downloaded from the project website. 

The solenoid and servo control in real time is a paramount consideration, especially in the 
real time use of the instruments. By using a computer keyboard to control the motors, one 
can actually improvise, and capture a spontaneous musical gesture. Such potential has 
theatrical implications. These improvisations can be recorded and saved as files, which can 
be played back with precision. A wonderful feature of this composition component is the 
ability to edit the file, essentially allowing for the "tweaking" of sequential detail. Such an 
option is yet another level of control in the musical compositional process. 
First, the composition component provides users with a direct programming environment. 
Based on a set of pre-defined syntax, users can create C++ programs which control each 
detailed motion of the robot musician, such as the bow manipulation of a string instrument 
and the drumstick or wand operation of a percussion instrument. Figure 5 shows two screen 
shots of Moth.C and its executing result. 

At the next level, the composition component allows users to compose music scores through the 
keyboard and mouse. Keys ranging from Q to P on the third line of the keyboard are currently 
being used to give commands for 10 instrument performance speed levels. 12 key-press states on 
the second line of keyboard facilitate a broad range of up to 12 solenoids control. For a string 
instrument, each key represents a solenoid string pressing position and produces a certain pitch. 
The mouse, as a hand-held device, controls bow movement for a string instrument. A simple 
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observation, for example, is that while the mouse scrolls leftward or rightward, the violin-bow 
moves forward or backward correspondingly. Meanwhile, speed-wise, the mouse ball and bow 
movement forms a positive correlation. Additionally, certain mouse-click combinations allow bow 
switching during the composition process, in robots such as the 2-string violin musician, Jasche. 
Finally, the composition component includes a recording function which detects and records 
details of each key-press or mouse-movement event, as well as the time delay in between. It 
introduces the Robot Musician Composition Format (RMCF), which is adopted during a music 
recording process. RMCF makes real-time manipulations more intuitive, allowing for greater 
musical expression. A composition adopting RMCF has lines of code; each line encodes a 
command to be sent to the control block. As an example, considering a 6.3534 seconds time delay 
between two bow movements (2 pitches, or two mouse movements), "*6.2524" is recorded with 
symbol "*" indicating time delay. In similar fashion to the time-delay recording, ""#4 Vl=150 @3 
Sll" means servo 1 of instrument number 4 moves to position 150 at a speed of level 3 with 
solenoid number 11 being pressed, which produces a pitch. RMCF brings new possibilities for 
users to directly edit those compositions, such as altering the time-delay between two notes, 
changing a note to a higher pitch, merging multiple instruments into one single composition by 
inserting command lines, etc. Figure 6 shows a recorded composition with RMCF. 
The performance component allows robot musician to read and play the musical scores 
recorded by the composition component or directly composed by a user. By single-line 
interpretation, the performance component is able to produce a musical experience that is 
almost identical to the original instrumental performance. 



* 1.318681 
##4S6 

* 0.000000 
##4 Vl=162 

* 0.274725 
##4 Vl=161 

* 0.000000 
##4 Vl=160 

* 0.000000 
##4 Vl=160 

* 0.054945 
##4 Vl=159 

* 0.219780 
##4 Vl=160 

* 0.164835 
##4S6 

* 0.000000 
##4 Vl=160 



Fig. 6. A recorded composition with RMCF. 

Along with the two major components mentioned above, the software module also includes 
a tuning component to help users adjust sound quality of the instruments before 
composition and performance. 



3.4 Software Module Infrastructure 

Architecturally, the control module consists of an RSV Converter and a costume- 
manufactured motion control card (Figure 7). The motion control card is built on an 
Aesthetika Robotika Servo Control Rev 4 board, powered by 12V and 500 mA (Figure 8). 
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The original board could support up to 16 servos, 16 solenoids and 4 steppers. It is custom 
manufactured for this project to support up to 4 servos (4 axes) and 12 solenoids. The 
converter interfaces between the serial port and motion control card (Figure 9). 
Jouve and Bui [3], state that the speed and position servo loop adjustment must be 
optimized according to the mechanical load characteristics in order to achieve a stable and 
fast response. For the case of a robot musician, the control module receives a list of motion 
control commands from the software module and computes joint angles and servo motor 
velocities for each axis. These values are then delivered to the motion module, ensuring 
smooth servo motor movements. 

The motion control card (MCC) supports high-level commands sent from the serial port of 
the computer terminal. These commands include operations on the solenoids (such as which 
one is to take action pressing the string), and the movement of the servos (such as which 
servo is to function). The functions that the rules govern closely mimic a single task or a 
series of tasks that a human musician performs. When a human musician's brain decides to 
move the bow of the violin forward 10 centimeters within 2 seconds, the central nervous 
system responds by issuing commands to the hand muscles, coordinating body movements 
to accomplish the task. In the case of a robot musician, the MCC serves as the central 
nervous system that ideally assures continuous motion of the robot hands. 




Fig. 7. Control module with motion control card and converter, linked with the motion 
module of an incomplete keyboard robot musician's hand. 




Fig. 8. Motion Control Card. 
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Fig. 9. Converter. 

3.5 Motion Module 

The Motion module (the robot hands) is provided by powered mechanics, namely servos and 
solenoids, which manage and provide access to the actual musical instruments. Two types of servos 
(Hitec Quarter Scale HS-700BB Servo and Hitec HS303 Servo) and three types of solenoids are used 
in its construction. The Shindengen solenoid has a holding power of 3 pounds, while Solenoids 
from Sun Magnet hold 1.5 pounds and those from Guardian Electric hold 0.75 pounds. All servos 
and solenoids are powered by 12 V (except the Shindengen solenoid, 24V) and 1500 mA. 




Fig. 10. Servo attached to one bow of Jasche. 




Fig. 11. Solenoid (with holding power of 1.5 pounds) attached to Jasche. 




Fig. 12. Solenoid (with holding power of 3 pounds) attached to Bernie Bot (A Cello Robot Musician). 
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Fig. 13. A coffee container's plastic lid is connected with a servo so it flutters against the 
body of a drum when the servo receives control command from the control module. 




Fig. 14. Sample Motion Module Architecture: violin bow controlled by servo, violin string 
pressed by solenoids. 




Fig. 15. Sample Motion Module Architecture: drumstick controlled by solenoid 




Fig. 16. Sample Motion Module Architecture: chimes wand controlled by servo. 

As an example, a violin is set up with two bows controlled by two servos and twelve fingers 
(solenoids). Servos are attached to bows of the violin. They move the bows back and forth 
across the violin strings, as shown in Figure 10. Solenoids, act as magnets to depress the 
strings when charged (Figure 11 and 12). 
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Figure 13 shows that a coffee container's plastic lid is connected with a servo so it flutters against 
the body of a drum when the servo receives control command from the control module. 
Figure 14, 15 and 16 shows three samples of motion module architecture. 



3.6 A Robot Musician Example: Jasche 

To give a more concrete view of the robot musician, a robot musician example is described 
in detail. The Robot Musician Jasche (Robot Musician No. 4) plays a 2-bow 12-finger 2-string 
violin tuned with 3 octave chromatic range (From F below middle C to two octaves above 
middle C), which produces 24 pitches. See Figure 17 for an overview picture of Jasche. 
The Software module for Jasche enables users to compose musical pieces through mouse 
movements. Mouse movements without mouse-click control the first bow. Mouse movements 
with left-mouse click manipulate the second bow. Keys on the third line of the keyboard, symbols 
"QWERTYUIOP" correspond to 10 speed levels of the bow movement. Keys on the second line of 
the keyboard, symbols "1234567890-=" correspond to 12 solenoids used to press the strings, 
producing different pitches. Composition adopting RMCF takes several forms, "##4 V2=163" 
means that the second bow of robot musician number 4 (Jasche) moves 163 unites of length away 
from its initial position; "* 0.494505" means that the computer pauses for 0.494505 seconds before 
sending the next command to the control module, i.e., there is a time delay of around 0.5 second 
between two continuous musical notes; "##4 @2" means that the bow movement has switched to 
speed level 2; "##4 S5" means that solenoids number 5 is being charged, thus it presses the string. 
Figure 18 shows a composition component for Jasche. Figure 19 shows a sample portion of 
composed musical piece for Jasche. 
There are several violin solos performed by Jasche available for download at the link provided later. 



Fig. 17. Jasche. 




Fig. 18. Composition program for Jasche. 
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##4 Vl=160 

* 0.879121 
##4 V2=163 

* 1.153846 
##4S5 

* 0.000000 
##4 Vl=163 

* 1.428571 
##4@2 

* 0.000000 
##4 V2=163 

* 0.494505 
##4@2 

* 0.000000 
##4 Vl=163 

* 0.659341 
##4S6 



Fig. 19. A sample portion of composed musical piece for Jasche. 



3.7 A Robot Musician Example: Jasche The Robot Musician Band Architecture 

In the construction of the P.A.M Band, 8 MCC boards are chained together, meaning, 8 robot 
musicians can be controlled by one single computer terminal simultaneously. The remaining 
4 robot musicians in the band are supported by two additional computers. A robot musician 
band architecture overview is illustrated in figure 20, where 8 robots share the same 
software module based on a single computer terminal (Figure 10). 
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Fig. 20. Robot Musician Band Architecture. 
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4. The Mechanical Challenges in Real Time Performance 

While more degrees of freedom can be achieved by a robot musician, the smooth transition 
between two musical notes (continuation of music score) involves the improvement of mobility, 
elasticity and sensitivity of the robot hands. Furthermore, high speed communication between the 
PC and the motion control card becomes critical during a real time music performance. Since time- 
delays between two musical notes are recorded by the software module only to a certain level of 
precision during the composition process, recording errors accumulate and become visible during 
the performance of a relatively lengthy composition. 



5. Results 

The recent work on robot musicians has some promising results. The existing system is 
found to be a robust method in controlling complex musical mechanics. The robotic musical 
piece " Mozart: Eine Kleine Nacht Musik" has exhibited a high-level behavior that exceeds 
initial expectations. Some images of the musical robots in action can be found in figure 21 
through figure 24. In order to better appreciate this work, several musical pieces performed 
by the robot musician band as well as some video clips are available for download at 
http://www.bridgeport.edu/sed/projects/IFACWeb/default.html 




Fig. 21. Gold member plays an electric guitar. 



Fig. 22. Stu plays a classical guitar 




Fig. 23. Zak plays an electric guitar. 
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Fig. 24. Drack plays an electric bass guitar. 

6. Robot Musical Instrument: A New Generation and a New Art Form 

The development of a robot musician introduces a new generation of robotic musical instruments, 
versus traditional instruments. Conceivably, designers might specifically construct musical 
instruments that are more complex in nature for robot musicians. These approaches could open up 
new domains for music expressivities. For example, a combinational drum set has already been set 
up for the robot musician, such that drums are specially positioned so that they are easily accessed 
by the robot hands (Figure 25). Furthermore, a violin can be played with two bows simultaneously. 




Fig. 25. Picture of Micky playing combinational drum set. 



Another interesting aspect, especially from a user's perspective, is the nature of the control he/she 
has with using programmable motors. The servos offer tremendous possibilities with respect to 
the violin. The violin uses the two hands of the performer in a very different, unrelated but 
integrated way. The left hand is for depressing the strings, making the different pitches; the right 
hand is for controlling the bow. The bow control technique is an art with a highly developed 
pedagogical tradition that evolved over hundreds of years, as the violin bow itself was modified 
from its original convex curve to the modern concave curve of the stick. New musical expressions 
and musical styles develop with the modifications in technique and equipment. The use of the 
servo offers new techniques, with recognizable innovation in bow control. For now, the left hand 
concept remains more directly linked to the tradition of "fretting" the string. Unlike a human- 
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played violin, the robot musician's fingers do not have any intonational inflection, which is one of 
the many limits machines suffer. However, it does open a whole new world of mirco-intonational 
possibilities. Fixed pitched instruments are typical in the percussion family, and the robotic violin 
seems to blur the distinction. 

What makes robot musicians interesting as a significant contribution to both art and 
technology, is the creation of a new art form, made possible by the use of a new technology 
in a simple, yet, until now, unexploited manner. Musical history can be reorganized by the 
incorporation of this new technology into the musical arts as a motivating force behind the 
development of new techniques and styles, as was the case with the invention of the cast- 
iron piano harp, steel wound violin strings, etc. 

Robot musicians feature a new application for the synchronized motion of servo motors and 
solenoids along with the use of traditional musical instruments in a new manner, which results in a 
new sonic environment. The repertoire is specifically developed and arranged for these particular 
instruments. The underlying aesthetic philosophy is not to replace the human performer, but rather, 
to explore unique musical expressions and expand the language of musical composition. 
Digitally-Manipulated Analogue Signal Systems (D-MASS) is proposed in this work as a 
description of this new art form. The following Table 2 makes a comparison between MIDI 
(Multi Instrument Digital Interface) and D-MASS. 





MIDI 


D-MASS 


Sound resources 


Triggers sound samples 
from a sound bank 


Analogue signal from an acoustic 
instrument 


System Requirement 


No mechanical devices 


Requires mechanical devices 


Sound quality 


Identical 


Allowing for subtle variances 
each event is unique 


Sound sample 
manipulation 


Samples manipulated 
electronically 


Acoustic waves can be 
manipulated electronically 



Table 2. Comparison table between MIDI and D-MASS. 

7. Future Developments 

In addition to the mechanical robot musician design, software/ hardware tools can be 
developed to enable robots to read and play traditional scores. This will open a new set of 
possibilities for automation and robotized musicians. 

Inspired by researches described in [4, 5], soft computing (fuzzy logic, neural networks, 
evolutionary and genetic algorithms) and artificial intelligence / expert systems techniques for 
programming mechanical devices will introduce an adaptive flavor for playing the instruments at 
a later stage of project development. Improving the emotional quality of the music performance 
and enabling the robot performers to communicate musically with their listeners will also be an 
interesting extension [6, 7, 8]. Furthermore, having the robot musicians listen to various music 
pieces, recognize the tones, improve on them and then re-play them is an important project goal. 
Wireless control perspectives for the mechanical/ musical devices are also feasible. 



8. Further Discussions 

We envision this work to be of significant value as it introduces unique and new paradigms 
in appreciating the interaction of music, computer science, robotics and the new concepts 
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that emerge from robot musicians. The analysis of the degree of intelligence and sensitivity 
achieved by robot musicians here constitutes an interpretation of how a musical piece can 
create a particular effect and subjective musical experience for the audience. 

9. Conclusions 

In conclusion, the research directions described above should make the experience of 
designing robot musicians a breath-taking experiment. We envision that a fully functional 
robot orchestra can be established in the near future, which will play not only simple tunes, 
but also masterpieces. Several instruments of the P.A.M. band have been exhibited publicly, 
including the 2002 Westchester Biennial at the Castle Gallery of the College of New Rochelle 
and the Silvermines Art Gallery in New Canaan CT. During the fall of 2002, the entire P.A.M 
Band was on exhibit in the Walter Austin Theatre at the University of Bridgeport, and was 
reported in the Connecticut Post, Stamford Advocate, and the Greenwich Times. In the 
spring of 2003, the instruments were be on exhibit at the M.A.T.A. Festival in New York. 
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1. Introduction 

Wheelless, limbless snake-like robots (Snakebots) feature potential robustness characteristics 
beyond the capabilities of most wheeled and legged vehicles - ability to traverse terrain that 
would pose problems for traditional wheeled or legged robots, and insignificant 
performance degradation when partial damage is inflicted. Moreover, due to their modular 
design, Snakebots may be cheaper to build, maintain and repair. Some useful features of 
Snakebots include smaller size of the cross-sectional areas, stability, ability to operate in 
difficult terrain, good traction, high redundancy, and complete sealing of the internal 
mechanisms (Dowling, 1997), (Worst, 1998). Robots with these properties open up several 
critical applications in exploration, reconnaissance, medicine and inspection. However, 
compared to the wheeled and legged vehicles, Snakebots feature (i) smaller payload, (ii) 
more difficult thermal control, (iii) more difficult control of locomotion gaits and (iv) inferior 
speed characteristics. 

Considering the first two drawbacks as beyond the scope of our work, and focusing on the 
issues of control and speed, we intend to address the following challenge: how to develop 
control sequences of Snakebot 7 s actuators, which allow for the fastest possible speed of 
locomotion achievable with Snakebot morphology. 

For many tasks and robot morphologies, it might be seen as a natural approach to handcraft 
the locomotion control code by applying various theoretical approaches (Burdick et al, 1993), 
(Hirose, 1993), (Zhang et al, 22). However, handcrafting might not be feasible for developing 
the control code of a real Snakebot due to its morphological complexity and the anticipated 
need of prompt adaptation under degraded mechanical abilities and/ or unanticipated 
environmental conditions. Moreover, while a fast locomotion gait might emerge from 
relatively simply defined motion patterns of morphological segments of Snakebot, the 
natural implication of the phenomenon of emergence in complex systems is that neither the 
degree of optimality of the developed code nor the way to incrementally improve the code is 
evident to the human designer (Morowitz, 2002). Thus, an automated, holistic approach for 
evaluation and incremental optimization of the intermediate solution(s) is needed (e.g. based 
on various models of learning or evolution in Nature) (Kamio et al, 2003), (Mahdavi & 
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Bentley, 2003), (Takamura et al, 2000). The proposed approach of employing genetic 
programming (GP) implies that the code, which governs the locomotion of Snakebot is 
automatically designed by a computer system via simulated evolution through selection and 
survival of the fittest in a way similar to the evolution of species in the nature (Koza, 1992), 
(Koza, 1994). The use of an automated process to design the control code opens the 
possibility of creating a solution that would be better than one designed by a human (Koza et 
al,2000). 

In principle, the task of designing the code of Snakebot could be formalized and the formal 
mathematical models incorporated into direct programmable control strategies. However, 
the eventual models would feature enormous complexity and such models are not 
recognized to have a known, analytically obtained exact optimal solution. The complexity of 
the model stems from the considerable amount of degrees of freedom of the Snakebot, which 
cannot be treated independently of each other. The dynamic patterns of the position, 
orientation, velocity vectors, and moreover, the points and times of contact with the surface 
(and consequently - the vectors of resulting traction forces, which propel the Snakebot) of 
each of the morphological segments of Snakebot has to be considered within the context of 
other segments. Furthermore, often the dynamic patterns of these parameters cannot be 
deterministically inferred from the desired velocity characteristics of the locomotion of 
Snakebot. Instead, the locomotion of the Snakebot is viewed as an emergent property at a 
higher level of consideration of a complex hierarchical system, comprising many relatively 
simply defined entities (morphological segments). In such systems the higher-level 
properties of the system and the lower-level properties of comprising entities cannot be 
induced from each other. GP (and evolutionary algorithms in general) is considered as an 
efficient way to tackle such ill-posed problems due to the ability of GP to find a near-optimal 
solution in a reasonable runtime. This ability often overcompensates the drawbacks of GP, 
which can be summarized as follows: 

(i) Relatively long runtime stemming from the significant computational effort (many 

potential solutions need to be evaluated before the sufficiently good solution is 

discovered) and poor computational performance (often the fitness evaluation is a 

time-consuming routine) of GP,. 

(ii) Non-determinism - the exact runtime needed to obtain the solution cannot be 

estimated in advance. Instead, a statistically obtained probability of success for 

various runtime intervals is applied as a characteristic of computational efficiency 

of GP. The non-determinism of GP is viewed as a natural consequence of the 

stochastic nature of both the way of creating the initial population of potential 

solutions to the problem and the genetic operations applied to evolve this 

population (crossover, mutation, and selection), and. 

(iii) Often the solution, automatically obtained via GP is quite complex and difficult to 

be comprehended by a human designer. Consequently, even simple man-made 

modification to such a solution is not a straightforward task. 

As an instance of evolutionary algorithms, Genetic algorithms (GA) differ from GP mainly in 

the genotypic representation (i.e. chromosome) of potential solutions. Instead of representing 

the solution as a computer program (usually - a parsing tree) featuring arbitrary structure 

and complexity as in GP, GA employs a fixed-length linear chromosome. This difference 

implies a favorable computational advantage of GA over GP for simple problems, because 

the linear chromosomes are computationally efficiently manipulated by genetic operations 

and interpreted by fitness evaluation routines. For complex tasks however (such as evolution 
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of locomotion gaits of Snakebot) the runtime overhead associated with the manipulation of 
genotype is negligible compared to the much more significant overhead of the fitness 
evaluation of the evolved (simulated or real) artifact in the (simulated or real) environment. 
Moreover, an efficient GA (in terms of computational effort, or number of fitness 
evaluations) often requires incorporation of extremely computationally heavy probabilistic 
learning models aimed at maintaining the complex inter-relations between the variables in 
the chromosome. In addition, the fixed-length chromosome usually implies that the latter 
comprises various, carefully encoded problem-domain-dependent parameters of the solution 
with an a priori known structure and complexity. This might be a concern when no such 
knowledge is available in advance, but rather needs to be automatically and autonomously 
discovered by the evolving artifact. The latter is especially true when the artifact has to 
perform in unanticipated, uncertain environmental conditions or under its own (possibly 
degraded) mechanical abilities. Evolving a Snakebot 7 s locomotion (and in general, behavior 
of any robot) by applying GP could be performed as a first step in the sequence of simulated 
off-line evolution (phylogenetic learning) on the software model, followed by on-line 
adaptation (ontogenetic learning) of evolved code on a physical robot situated in a real 
environment (Meeden & Kumar, 1998). Off-line software simulation facilitates the process of 
Snakebot 7 s controller design because the verification of behavior on a physical Snakebot is 
extremely time consuming, costly and often dangerous for the Snakebot and surrounding 
environment. Moreover, in some cases it is appropriate to initially model not only the 
locomotion, but also to co-evolve the most appropriate morphology of the artifact (i.e. 
number of phenotypic segments; types and parameters of joints which link segments; 
actuators' power; type, amount and location of sensors; etc.) (Sims, 1994), (Ray, 2001) and 
only then (if appropriate) to physically implement it as hardware. The software model used 
to simulate Snakebot should fulfill the basic requirements of being quickly developed, 
adequate, and fast running (Jacobi, 1998). The typically slow development time of GP stems 
from the highly specific semantics of the main attributes of GP (e.g. representation, genetic 
operations, and fitness evaluation) and can be significantly reduced through incorporating 
off-the-shelf software components and open standards in software engineering. To address 
this issue, we developed a GP framework based on open XML standard. And to ensure 
adequacy and runtime efficiency of the Snakebot simulation we applied the Open Dynamic 
Engine (ODE) freeware software library for simulation of rigid body dynamics. 
The objectives of our work are (i) to explore the feasibility of applying GP for automatic design 
of the fastest possible, robust, general and adaptive locomotion gaits of realistically 
simulated Snakebots and (ii) to investigate the emergent properties of these locomotion gaits. 
Inspired by the fast sidewinding locomotion of the rattlesnake Crotalus cerastes, this work is 
motivated by our desires (i) to model the sidewinding locomotion of natural snakes, (ii) to 
explore the phenomenon of emergence of locomotion of complex bodies from simply defined 
morphological segments comprising these bodies. The remainder of this document is 
organized as follows. Section 2 emphasizes the main features of the GP proposed for 
evolution of locomotion gaits of the simulated Snakebot. Section 3 presents empirical results 
of emergent properties of evolving locomotion gaits of Snakebot and discusses robustness, 
generality and adaptation of sidewinding in various fitness landscapes caused by various, 
unanticipated environmental conditions and partial damage of Snakebot. The same section 
explores the relevant practical implications of the observed analogy between the emergent 
properties of the robust and adaptive sidewinding locomotion gaits of Snakebot. Finally, 
Section 4 draws conclusions. 




562 Mobile Robots, Towards New Applications 



2. The Approach 

2.1 Representation of Snakebot 

Snakebot is simulated as a set of identical spherical morphological segments ( // vertebrae ,/ ), 
linked together via universal joints. All joints feature identical finite angle limits and each 
joint has two attached actuators ("muscles"). In the initial standstill position, the rotation 
axes of the actuators are oriented vertically (vertical actuator) and horizontally (horizontal 
actuator) and perform rotation of the joint in the horizontal and vertical planes respectively 
(Figure 1). The task of designing the Snakebot locomotion can be rephrased as developing 
temporal patterns of desired turning angles of horizontal and vertical actuators of each 
segment, that result in the fastest overall locomotion. 



V«fiwl plane 

m 

Fig. 1. Morphological segments of Snakebot linked via universal joint. Horizontal and 

vertical actuators attached to the joint perform rotation of the segment #i+l in vertical and 
horizontal planes respectively. 

The proposed representation of Snakebot as a homogeneous system comprising identical 
morphological segments is intended to significantly reduce the size of the search space of the 
GP. Moreover, because the size of the search space does not necessarily increase with the 
increase of the number of morphological segments of Snakebot, the proposed approach 
allows achievement of favorable scalability characteristics of the GP. 

An alternative approach of employing phase automata has been recently proposed for 
representing and programming the functionality of segments in modular chain-type artifacts 
(Zhang et al., 2003). The approach is based on an event-driven input/ output state automaton 
with an initial phase delay, and promises great versatility, robustness and scalability. 
However, the eventual automatic programming of these locomotion gaits (rather than 
handcrafting them) is still an open issue in this approach. 

In the proposed representation of Snakebot, no anisotropic (directional) friction between the 
morphological segments and the surface is considered. Despite the anticipated ease of 
simulation and design of eventual morphological segments featuring anisotropic friction 
with the surface (using simple attached wheels (Hirose, 1993) or "belly" scales), such an 
approach would have the following drawbacks: 

(i) Wheels, attached to the morphological segments of Snakebot are mainly effective in 
two-dimensional locomotion gaits. However, neither the fastest gaits in 
unconstrained environments nor the adaptive gaits in challenging environments 
(narrow tunnels, obstacles etc.) are necessarily two-dimensional. In 
three-dimensional locomotion gaits the orientation (the pitch, roll and yaw angles) 
of morphological segments at the instant of contact with the surface are arbitrary, 
which renders the design of effective wheels for such locomotion gaits a non-trivial 
engineering task, 
(ii) Wheels may compromise the potential robustness characteristics of Snakebot 
because they can be trapped easily in the challenging environments (rugged terrain, 
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obstacles, etc.). 
(iii) Wheels potentially reduce the application areas of the Snakebot because their 

engineering design implies lack of complete sealing of all mechanisms of Snakebot. 
(iv) Belly scales (if implemented) would not promote any anisotropic friction when 

Snakebot operates on smooth, flat, clean and/ or too loose surfaces which 

compromises the generality of derived locomotion gaits and their robustness to 

various environmental conditions. 
Belly scales are efficiently utilized as a source of anisotropic friction in some locomotion gaits 
of natural snakes. However, these gaits usually require an involvement of a large amount of 
complex muscles located immediately under the skin of the snake. These muscles lift the 
scales off the ground, angle them forward, and then push them back against the surface. In 
the Snakebot, implementing actuators, which mimic such muscles in the natural snakes, 
would be expensive and thus infeasible from an engineering point of view. 

2.2 Algorithmic Paradigm 

2.2.1 GP 

GP (Koza, 1992), (Koza, 1994) is a domain-independent problem-solving approach in which a 
population of computer programs (individuals 7 genotypes) is evolved to solve problems. The 
simulated evolution in GP is based on the Darwinian principle of reproduction and survival 
of the fittest. The fitness of each individual is based on the quality with which the phenotype 
of the simulated individual is performing in a given environment. The major attributes of GP 
- function set, terminal set, fitness evaluation, genetic representation, and genetic operations 
are elaborated in the remainder of this Section. 

2.2.2. Function Set and Terminal Set 

In applying GP to the evolution of Snakebot, the genotype is associated with two algebraic 
expressions, which represent the temporal patterns of desired turning angles of both the 
horizontal and vertical actuators of each morphological segment. Since locomotion gaits are 
periodical, we include the trigonometric functions sin and cos in the GP function set in 
addition to the basic algebraic functions. The choice of these trigonometric functions reflects 
our intention to verify the hypothesis, first expressed by Petr Miturich in 1920's (Andrusenko, 
2001), that undulative motion mechanisms could yield efficient gaits of snake-like artifacts 
operating in air, land, or water. 

From another perspective, the introduction of sin and cos in the function set of GP reflects 
our intention to mimic (at functional, rather than neurological level) to some extent the 
central pattern generator (CPG) in the central nervous system (usually located in the ganglia 
or spinal cord) of animals, believed to be necessary and sufficient for the generation of 
rhythmic patterns of activities. CPG for robot control typically comprises coupled neural 
controllers, which generate (without the need of external feedback) the motion pattern of 
actuators in the respective morphological segments of the artifact. The approach of 
employing CPG for developing the locomotion gaits of the Snakebot would be based on an 
iterative process (e.g. employing the machine learning and/ or evolutionary computations 
paradigms) of tuning the main parameters of CPG including, for example, the common 
single frequency across the coupled oscillators, the fixed phase-relationship between the 
oscillators, and the amplitude of each of the oscillations. The proposed approach of applying 
GP for evolution of locomotion gaits of Snakebot shares some of the features of CPG-based 
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approaches such as the open-loop control scheme and the incorporation of coupled 
oscillators. Conversely to the CPG-based approaches however, the proposed method 
incorporates too little domain-specific knowledge about the task. The comparative flexibility 
of GP, resulting from not considering all the domain-specific constrains, can potentially yield 
an optimal solution with the following properties, typically uncommon for CPG: 

(i) The optimal oscillations of segments might be an arbitrary superposition of several 

oscillations featuring different frequencies. Moreover, the proposed method of 

using GP does not necessarily imply that the frequency across the oscillators is 

common, 
(ii) The relationship between the oscillators in the morphological segments of Snakebot 

is not necessarily a simple phase relationship. Arbitrary relationships involving 

amplitude, phase, and frequency are allowed to be developed by the simulated 

evolution via GP, and 
(iii) The evolved optimal phase relationship between the oscillators in the 

morphological segments might vary along the body of Snakebot, rather than being 

fixed. 
The above-mentioned features are achieved via the incorporation of the terminal symbol 
segment_ID (an unique index of morphological segments of Snakebot), which allows GP to 
discover how to specialize (by phase, amplitude, frequency etc.) the temporal motion patterns 
(i.e. the turning angles) of actuators of each of the identical morphological segments of the 
Snakebot. In addition, the terminal symbols of GP include the variables time and two 
constants: Pi, and random constant within the range [0, 2]. The introduction of variable time 
reflects our objective to develop temporal patterns of turning angles of actuators. The main 
parameters of the GP are summarized in Table 1. 



Category 


Value 


Function set 


{sin, cos, +, -, *, /, ADF} 


Terminal set 


{time, segment ID, Pi, random constant } 


Population size 


200 individuals 


Selection 


Binary tournament, ratio 0.1 


Elitism 


Best 4 individuals 


Mutation 


Random subtree mutation, ratio 0.01 


Fitness 


Velocity of simulated Snakebot during the trial. 


Trial interval 


180 time steps, each time step account for 50ms of "real" 


time 


Termination 
criterion 


(Fitness > 1 00) or (Generations > 40) or (no improvemen 
for 16 generations) 


t of fitness 



Table 1. Main Parameters of GP. 

The rationale of employing automatically defined functions (ADF) is based on the empirical 
observation that the evolvability of straightforward, independent encoding of desired 
turning angles of both horizontal and vertical actuators is poor, although it allows GP to 
adequately explore the potentially large search space and ultimately, to discover the areas 
which correspond to fast locomotion gaits in solution space. We discovered that (i) the 
motion patterns of horizontal and vertical actuators of each segment in fast locomotion gaits 
are highly correlated (e.g. by frequency, direction, etc.) and that (ii) discovering and 
preserving such correlation by GP is associated with enormous computational effort. ADF, as 
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a way of limiting the search space by introducing modularity and reuse of code in GP (Koza, 
1994) is employed in our approach to allow GP to explicitly evolve the correlation between 
motion patterns of horizontal and vertical actuators as shared fragments in algebraic 
expressions of desired turning angles of actuators. Moreover, we observed that the best 
result is obtained by (i) allowing the use of ADF in the algebraic expression of desired 
turning angle of vertical actuator only, and (ii) by evaluating the value of ADF by equalizing 
it to the value of the currently evaluated algebraic expression of the desired turning angle of 
the horizontal actuator. 

2.2.3 Fitness evaluation 

The fitness function is based on the velocity of Snakebot, estimated from the distance which the 
center of the mass of Snakebot travels during the trial. The energy consumed by the Snakebot 
during the trial is not considered in our work. The real values of the raw fitness, which are 
usually within the range (0, 2) are multiplied by a normalizing coefficient in order to deal with 
integer fitness values within the range (0, 200). A normalized fitness of 100 (one of the 
termination criteria shown in Table 1) is equivalent to a velocity, which displaced Snakebot a 
distance equal to twice its length. The fitness evaluation routine is shown in Figure 2. 

2.2.4 Representation of genotype 

The slight performance degradation in computing the desired turning angles of actuators by 
traversing the DOM/ XML-based representation of genetic programs during fitness 
evaluation is not relevant for the overall performance of GP. The performance profiling 
results indicate that the fitness evaluation routine consumes more than 99% of the GP 
runtime. However, even for relatively complex genetic programs featuring a few hundred 
tree nodes, most of the fitness evaluation runtime at each time step is associated with the 
relatively large computational cost of the physics simulation (actuators, joint limits, friction, 
gravity, collisions, etc.) of phenotypic segments of the simulated Snakebot (routine 
dWorldStep in Figure 2, line 20), rather than computing the desired turning angles of 
actuators. Inspired by its flexibility, and the recent widespread adoption of the document 
object model (DOM) and extensible markup language (XML), we represent evolved 
genotypes of simulated Snakebot as DOM-parse trees featuring equivalent flat XML-text, as 
first discussed by (Tanev, 2004). Our approach implies that both (i) the calculation of the 
desired turning angles during fitness evaluation (function EvalDesiredAngle, shown in 
Figure 2, line 15) and (ii) the genetic operations are performed on DOM-parse trees using 
off- the shelf, platform and language neutral DOM-parsers. The corresponding XML-text 
representation (rather than S-expression) is used as a flat file format, feasible for migration of 
genetic programs among the computational nodes in an eventual distributed implementation 
of the GP. The benefits of using DOM/ XML-based representations of genetic programs are 
(i) fast prototyping of GP by using standard built-in API of DOM-parsers for traversing and 
manipulating genetic programs, (ii) generic support for the representation of grammar of 
strongly-typed GP using W3C-standardized XML-schema; and (iii) inherent 
Web-compliance of eventual parallel distributed implementation of GP. 

2.2.5 Genetic operations 

We employ binary tournament selection - a robust, commonly used selection mechanism, 
which has proved to be efficient and simple to code. The crossover operation is defined in a 
strongly typed way in that only the DOM-nodes (and corresponding DOM-subtrees) of the 
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same data type (i.e. labeled with the same tag) from parents can be swapped. The sub-tree 
mutation is allowed in a strongly typed way in that a random node in the genetic program is 
replaced by a syntactically correct sub-tree. The mutation routine refers to the data type of 
the currently altered node and applies a randomly chosen rule from the set of applicable 
rewriting rules as defined in the grammar of the GP. 



I. function Evaluate (GenH, GenV: TGenotype): real; 

2.//GenH and GenV is the evolved genotype: a pair of algebraic 

3. //expressions, which define the turning angle of horizontal 

4. //and vertical actuators at the joints of Snakebot. 

5. Const 

6. TimeSteps =180; // duration of the trial 

7. SegmentsInSnakebot=15;//# ofphenotypic segments 

8. var 

9. t, s : integer; 

10. AngleH, AngleV : real m //desired turning angles of actuators 

II. CAngleH, CAngleV: real;//current turning angles of actuators 

12. InitialPos, FinalPos: 3DVector;// (X,Y,Z) 

13. begin 

14. InitialPos:=GetPosOfCenterOfMassOfSnakebot; 

15. for t:=0 to TimeSteps-1 do begin 

16. for s:=0 to SegmentsInSnakebot-1 do begin 

17. //traversing XML/DOM-based GenH using DOM-parser: 

18. AngleH := EvalHorisontalAngle(GenH,s,t); 

19. //traversing XML/DOM-based GenV using DOM-parser: 

20. AngleV := E val Vertical Angle (GenV, s,t); 

21. CAngleH := GetCurrentAngleH(s); 

22. CAngleV := GetCurrentAngleV(s); 

23. SetDesiredVelocityH(CAngleH-AngleH,s); 

24. SetDesiredVelocityV(CAngleV-AngleV,s); 

25. end; 

26. //detecting collisions between the objects (phenotypic 

27 .// segments, ground plane, etc.): 

28. dSpaceCollide; 

29 .// Obtaining new properties (position, orientation, 

30. //velocity vectors, etc.) of morphological segments of 

31. //Snakebot as a result of applying all forces: 

32. dWorldStep; 

33. end; 

34. FinalPos := GetPosOfCenterOfMassOfSnakebot; 

35. return GetDistance (InitialPos, FinalPos)/ (TimeSteps); 

36. end; 



Fig. 2. Fitness evaluation routine. 



2.2.6 ODE 

We have chosen Open Dynamics Engine (ODE) (Smith, 2001) to provide a realistic 
simulation of physics in applying forces to phenotypic segments of Snakebot, for 
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simulation of Snakebot locomotion. ODE is a free, industrial quality software library for 
simulating articulated rigid body dynamics. It is fast, flexible and robust, and it has 
built-in collision detection. The ODE-related parameters of the simulated Snakebot are 
summarized in Table 2. 



Parameter 


Value 


Number of phenotypic segments in snake 


15 


Model of the segment 


Sphere 


Radius of the segment, cm 


3 


Overlap between segments, % 


25 


Length of the Snakebot, cm 


66 


Volume of the segment, cm3 


113 


Density of the segment, g/cm3 


0.9 


Mass of the segment, g 


100 


Type of joint between segments 


Universal 


Initial alignment of segments in Snakebot 


Along Y-axis of the world 


Number of actuators per joint 


2 


Orientation of axes of actuators 


Horizontal - along X-axis and Vertical - along 
Z-axis of the world 


Operational mode of actuators 


dAMotorEuler 


Max torque of actuators, gem 


12000 



Table 2. ODE-related parameters of simulated Snakebot. 



3. Empirical Results 

This section presents the experimental results verifying the feasibility of applying GP for 
evolution of the fast locomotion gaits of Snakebot for various fitness and environmental 
conditions. In addition, it investigates the emergent properties of (i) the fastest locomotion 
gaits, evolved in unconstrained environmental conditions and (ii) the robust locomotion 
gaits evolved in challenging environments. The section also discusses the gradual adaptation 
of the locomotion gaits to degraded mechanical abilities of Snakebot. These challenges are 
considered as relevant for successful accomplishment of various practical tasks during 
anticipated exploration, reconnaissance, medicine and inspection missions. In all of the cases 
considered, the fitness of Snakebot reflects the low-level objective (i.e. what is required to be 
achieved) of Snakebot in these missions, namely, to be able to move fast regardless of 
environmental challenges or degraded abilities. The experiments discussed illustrate the 
ability of the evolving Snakebot to learn how (e.g. by discovering beneficial locomotion traits) 
to accomplish the required objective without being explicitly taught about the means to do 
so. Such know-how acquired by Snakebot automatically and autonomously can be viewed as a 
demonstration of emergent intelligence in that the task-specific knowledge of how to 
accomplish the task emerges in the Snakebot from the interaction of the problem solver and a 
fitness function (Angeline, 1994). 
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3.1 Emergent Properties of Evolved Fastest Locomotion Gaits 

Figure 3 shows the fitness convergence characteristics of 10 independent runs of GP 
(Figure 3a) and sample snapshots of evolved best-of-run locomotion gaits (Figure 3b and 
Figure 3c) when fitness is measured regardless of direction in an unconstrained 
environment. Despite the fact that fitness is unconstrained and measured as velocity in 
any direction, sidewinding locomotion (defined as locomotion predominantly 
perpendicular to the long axis of Snakebot) emerged in all 10 independent runs of GP, 
suggesting that it provides superior speed characteristics for Snakebot morphology. The 
evolved locomotion gait is quite similar to the locomotion of the natural snake Crotalus 
cerastes, or // sidewinder ,/ . In the proposed representation of Snakebot, similarly to the 
natural snake, no anisotropic (directional) friction between the morphological segments 
and the surface is considered. Consequently, no forward locomotion (which requires an 
extensive utilization of anisotropic friction) emerges as a locomotion that is fast enough to 
challenge the achieved velocity of sidewinding. 
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Fig. 3. Evolution of locomotion gaits for cases where fitness is measured as velocity in any 
direction. Fitness convergence characteristics of 10 independent runs (a), probability of 
success (i.e., probability of attaining a fitness value of 100) (b), and snapshots of sample 
evolved best-of-run sidewinding locomotion gaits of simulated Snakebot (c). The dark 
trailing circles in (c) depict the trajectory of the center of the mass of Snakebot. Timestamp 
interval between each of these circles is fixed and it is the same (10 time steps) for both 
snapshots. 

The genotype of a sample best-of-run genetic program is shown in Figure 4. The dynamics of 
evolved turning angles of actuators in sidewinding locomotion result in characteristic 
circular motion pattern of segments around the center of mass as shown in Figure 5a. The 
circular motion pattern of segments and the characteristic track on the ground as a series of 
diagonal lines (as illustrated in Figure 5b) suggest that during sidewinding the shape of 
Snakebot takes the form of a rolling helix (Figure 5c). Figure 5 demonstrates that the 
simulated evolution of locomotion via GP is able to invent the improvised "cylinder" of the 
sidewinding Snakebot to achieve fast locomotion. 

By modulating the oscillations of the actuators along the snake's body, the diameter of the 
cross-section of the "cylinder" can be tapered towards either the tail or head of the snake, 
providing an efficient way of "steering" the Snakebot (Figure 6a, 6b). We consider the 
benefits of modulating the oscillations of actuators along the body of Snakebot as a 
straightforward implication of our understanding of the emergent form of a rolling helical 
Snakebot. Such modulation is implemented as a handcrafted (rather than evolved) feature of 
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evolved best-of-run sidewinding Snakebots. Snapshots, shown in Figure 6c and 6d illustrate 
the ability of Snakebot to perform sharp turns with a radius similar to its length in both 
clockwise and counterclockwise directions. 



GenH = (sin(((sin(-8)) * (segment id - time)) + (3 * time)))/(sin(-8)); 
GenV = sin(ADF) 



Fig. 4. Normalized algebraic expressions of the genotype of a sample best-of-run genetic 
program: dynamics of turning angle of horizontal (GenH) and vertical (GenV) actuators. The 
value of the automatically defined function ADF, in GenV, is evaluated by equalizing it to the 
value of the currently evaluated GenH. 
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Fig. 5. Trajectory of the central segment (cs) around the center of mass (cm) of Snakebot for 
a sample evolved best-of-run sidewinding locomotion (a), traces of ground contacts (b), and 
Snakebot, wrapped around an imagined cylinder taking the form of a rolling helix. 
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Fig. 6. Steering the Snakebot. The Snakebot moving straight is wrapped around an imagined 
cylinder taking the form of a rolling helix (a). By modulating the oscillations of the actuators 
along the snake's body, the diameter of the cross-section of the // cylinder ,/ can be tapered 
towards either the tail or head of the snake, providing an efficient way of // steering ,/ the 
Snakebot: (b) illustrates the Snakebot turning counterclockwise. The images in (a) and (b) are 
idealized: in the simulated Snakebot (and in snakes in Nature too) the cross-sectional areas of 
the imagined "cylinder" (a) and // cone ,/ (b) are much more similar to ellipses (as shown in 
Figure 5a) rather than to perfect circles as depicted here. The snapshots shown in (c) and (d) 
illustrate the Snakebot performing sharp turns in both clockwise and counterclockwise 
directions, respectively. 

In order to verify the superiority of velocity characteristics of sidewinding locomotion for 
Snakebot morphology we compared the fitness convergence characteristics of evolution in an 
unconstrained environment for the following two cases: (i) unconstrained fitness measured 
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as velocity in any direction (as discussed above), and (ii) fitness, measured as velocity in the 
forward direction only. The results of evolution of forward locomotion, shown in Figure 7, 
indicate that non-sidewinding motion features much inferior velocity characteristics, 
compared to sidewinding. 
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Fig. 7. Evolution of locomotion gaits for cases where fitness is measured as velocity in the 
forward direction only: fitness convergence characteristics of 10 independent runs of GP (a) 
and snapshots of sample best-of-run forward locomotion (b). Timestamp interval between 
the traces of the center of the mass is the same as for sidewinding locomotion gaits, shown in 
Figure 3c. 

The results of evolution of rectilinear locomotion of the simulated Snakebot confined in a 
narrow "tunnel" are shown in Figure 8. The width of the tunnel is three times the diameter of 
the cross-section of the segment of Snakebot. Compared to forward locomotion in an 
unconstrained environment (Figure 7), the velocity in this experiment is superior, and 
comparable to the velocity of sidewinding (Figure 3). This, seemingly anomalous emergent 
phenomenon demonstrates the ability of simulated evolution to discover a way to utilize the 
walls of the "tunnel" as a source of (i) extra grip and (ii) locomotion gaits (e.g., vertical 
undulations) which are fast yet unbalanced in an unconstrained environment. Indeed, as 
soon as the Snakebot clears the tunnel, the gait flattens (Figure 8c) and velocity (visually 
estimated as a distance between the traces of the center of gravity of Snakebot) drops 
dramatically. 
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Fig. 8. Evolution of locomotion gaits of Snakebot when confined in a narrow "tunnel": 
fitness convergence characteristics of 10 independent runs of GP (a) and snapshots of sample 
evolved best-of-run gaits at the intermediate (b) and final stages of the trial (c). 
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3.2 Robustness via Adaptation to Challenging Environment. Generality of the Evolved 
Robust Gaits 

Adaptation in Nature is viewed as an ability of species to discover the best phenotypic (i.e. 
pertaining to biochemistry, morphology, physiology, and behavior) traits for their survival in a 
continuously changing fitness landscape. Adaptive phenotypic traits are the result of beneficial 
genetic changes which occurred in due course of the evolution (phylogenesis) and/ or 
phenotypic plasticity (ontogenesis - learning, polymorphism, polyphenism, immune response, 
adaptive metabolism, etc.) occurring during the lifetime of the individuals. In our approach we 
employ GP for adaptation of Snakebot to changes in the fitness landscape caused by (i) a 
challenging environment and (ii) partial damage to 1, 2, 4 and 8 (out of 15) morphological 
segments. The former case is discussed in this subsection, while the latter case is elaborated in 
the following subsection 3.3. In both cases of adaptation, GP is initialized with a population 
comprising 20 best-of-run genetic programs, obtained from 10 independent evolutionary runs 
in unconstrained environments, plus an additional 180 randomly created individuals. 
The challenging environment is modeled by the introduction of immobile obstacles 
comprising 40 small, randomly scattered boxes, a wall with height equal to 0.5 diameters of 
the cross-section of Snakebot, and a flight of 3 stairs, each with height equal to 0.33 diameters 
of the cross-section of Snakebot. Both the wall and the stairs have a finite length. However, 
because no feedback from the environment to steer the snakebot is employed in our 
experiment, any attempt of the Snakebot to bypass the wall would lead to a sort of sustained 
cycling trajectories of the snakebot. However, these trajectories would be discouraged by the 
simulated evolution because they feature inferior distance between the position of the 
snakebot at the start and the finish of the trial, and consequently, inferior fitness values. The 
fitness of adapting Snakebot is measured in any direction. 

The empirical results of adaptation of the sidewinding Snakebot, obtained over 10 independent 
runs reveal the poor performance of the best-of-run Snakebots initially evolved in 
unconstrained environments. The fitness of the best-of-run Snakebots immediately drops from 
initial value of 100 in the unconstrained environment to only 65 when Snakebot is first tested (at 
Generation #0) on the challenging terrain, which indicates the poor initial robustness of these 
locomotion gaits. However, adapting to the new environment, the evolving Snakebots are able 
to discover locomotion gaits which are robust enough to allow the Snakebots to overcome the 
various kinds of obstacles introduced in the environment. About 20 generations of 
computational effort is required to reach fitness values of 100 in the challenging environment 
with probability of success 0.9. Snapshots illustrating the performance of a sample best-of-run 
Snakebot initially evolved in unconstrained environment, before and after the adaptation to the 
challenging environment are shown in Figure 9. 

The emergent properties of the robust sidewinding gaits are shown in Figure 10. As depicted 
in the Figure, the additional elevation of the body, required to negotiate the obstacles faster 
represents the emergent know-how in the adapting Snakebot. As shown in Figure lOe, the 
trajectory of the central segment around the center of the mass of sample adapted Snakebot is 
almost twice as high as before the adaptation (Figure 5a). Moreover, as the snapshots of the 
adapted gaits of Snakebot viewed from the above (Figure 10b and lOd) reveal, the robust 
locomotion gaits are associated with much higher winding angle of locomotion (about 120°) 
yielding longitudinally more compact sidewinding Snakebots. Again, as with the emergence 
of sidewinding, the result of the artificial evolution is analogous to the solution discovered 
by Nature - it is recognized that natural snakes also change the winding angle of the 
locomotion in order to adapt themselves to the various environmental conditions. 
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The generality of the evolved robust sidewinding gaits is demonstrated by the ease with 
which Snakebot, evolved in known challenging terrain overcomes various types of 
unanticipated obstacles such as a pile of or burial under boxes, and small walls, as illustrated 
in Figures 11, 12, and 13. 








Fig. 9. Snapshots illustrating the sidewinding Snakebot, initially evolved in unconstrained 
environment, before the adaptation - initial (a), intermediate (b and c) and final stages of the 
trial (d), and after the adaptation to challenging environment via GP - initial (e), intermediate 
(f) and final stages of the trial (g). The challenging environment is modeled by the 
introduction of immobile obstacles comprising 40 small, randomly scattered boxes, a wall 
with height equal to 0.5 diameters of the cross-section of Snakebot, and a flight of 3 stairs, 
each with height equal to 0.33 diameters of the cross-section of Snakebot. 
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Fig. 10. Snapshots of frontal view (a, c) and view from the above (b, d) of sample sidewinding 
Snakebots before and after the adaptation, respectively. The frontal views (a and c) 
comparatively illustrates the additional elevation of the body of the adapted Snakebot. The 
trajectory of the central segment (cs) around the center of mass (cm) of Snakebot for sample 
best-of-run sidewinding locomotion after the adaptation (e) to challenging environment 
indicates that the elevation of the central segment after the adaptation (a) is twice as high as 
before the adaptation (as illustrated in Figure 5a). 
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Fig. 11. Snapshots illustrating the generality of sidewinding Snakebot adapted to the 
challenging environment depicted in Figure 9. Before the adaptation to the challenging 
environment the Snakebot overcomes an unanticipated pile of boxes slower (a, b and c) than 
after the adaptation (d, e, and f). 
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Fig. 12. Snapshots illustrating the generality of sidewinding Snakebot adapted to the 
challenging environment depicted in Figure 9. Before the adaptation to the challenging 
environment the Snakebot emerges from an unanticipated burial under a pile of boxes slower 
(a, b and c) than after the adaptation (d, e, and f). 
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Fig. 13. Snapshots illustrating the generality of sidewinding Snakebot adapted to the 
challenging environment depicted in Figure 9. Before the adaptation to the challenging 
environment the Snakebot clears unanticipated walls forming a pen slower (a, b, c and d) 
than after the adaptation (e, f and g). The walls are twice as high as in the challenging 
terrain of Figure 9, and their height is equal to the diameter of the cross-section of 
Snakebot. 



3.3 Adaptation to Partial Damage 

The adaptation of sidewinding Snakebot to partial damage to 1, 2, 4 and 8 (out of 15) 
segments by gradually improving its velocity is shown in Figure 14. Demonstrated 
results are averaged over 10 independent runs for each case of partial damage to 1, 2, 4 
and 8 segments. The damaged segments are evenly distributed along the body of 
Snakebot. Damage inflicted to a particular segment implies a complete loss of 
functionality of both horizontal and vertical actuators of the corresponding joint. As 
Figure 14 depicts, Snakebot quickly and completely recovers from damage to a single 
segment, attaining its previous velocity only in 7 generations. Also in the case of 2 
damaged segments, Snakebot recovers to an average of 100% of its previous velocity. 
With 4 and 8 damaged segments the degree of recovery is 92% and 72% respectively. 
The emergent properties of adapted sidewinding locomotion gaits are shown in Figure 
15. 
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Fig. 14. Adaptation of Snakebot to damage of 1 (a), 2 (b), 4 (c) and 8 (d) segments, respectively. 
The results are averaged over 10 independent runs of GP for the cases when GP is initialized 
with the best-of-run Snakebots evolved in unconstrained and challenging terrain 
respectively. 
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Fig. 15. The emergent properties of adapted sidewinding locomotion gaits: frontal view of 
the Snakebot before (a) and after the adaptation (b) to the damage of a single segment 
demonstrates the additional elevation of the adapted Snakebot. View of the shape of the 
sidewinding Snakebot from above reveals the emergent tendency of increasing the winding 
angle of locomotion in a way similar to adaptation to the challenging environment (as shown 
in Figure 10): Snakebot with 1 (c, d), 2 (e, f), 4 (g, h) and 8 (i, j) damaged segments before and 
after the adaptation, respectively. 



3.4 Genetic Similarity of Adapted Snakebots 

In order to investigate whether the analogy in the emergent properties of locomotion gaits 
result from similar genotypes, we analyzed the correlation between the frequencies of 
occurrence of tree nodes in a particular context (i.e. the parent- and the descendant tree 
nodes) in the genetic representations of the three categories of Snakebots - (i) evolved in 
smooth unconstrained environment, (ii) adapted to challenging environment, and (ii) 
adapted to the degraded mechanical abilities - as elaborated earlier in Sections 3.1, 3.2 and 
3.3 respectively. For each of these three categories of Snakebots we aggregated the 
frequencies of occurrence of tree nodes obtained from the genotypes of the 20 best-of-run 
Snakebots (from 10 independent runs of GP). The results are as follows: 

(i) The correlation between genotypes of the Snakebots evolved in smooth 

environment and the Snakebots adapted to challenging environment is Cs-c = 0.34, 
(ii) The correlation between genotypes of the Snakebots evolved in smooth 

environment and the Snakebots adapted to degraded mechanical abilities due to 

partial damage is Cs-d = 0.32, and. 
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(iii) The correlation between genotypes of the Snakebots adapted to challenging 
environment and the Snakebots adapted to degraded mechanical abilities due to 
partial damage is Cc-d=0.91. 
These results suggest that there is little similarity between the genotypes of Snakebots 
adapted to both changes in the fitness landscape (i.e., due to challenging environment 
and partial damage) and the Snakebot evolved in smooth environment. We assume that 
this limited similarity (Cs-c=0.34, and Cs-d=0.32) is due to the shared genotypic fragments, 
which are relevant for the very ability of Snakebot to move, regardless of the 
environmental conditions and/ or the mechanical failures. These results also show that in 
both cases the genotype of Snakebots adapts to changes in the fitness landscape by 
drifting away from the genotype of the common ancestor - the Snakebot evolved in 
smooth environment, used to initially feed the adapting populations of Snakebots. 
Moreover, the strong correlation between the genotypes of adapted Snakebots 
(Cc-d = 0.91) suggests that the adaptation in both cases is achieved through a drift towards 
adjacent niches in the genotypic space of the Snakebot. This, in turn, yields the 
discovered phenotypic analogy between the adapted Snakebots, as discussed above in 
Sections 3.2 and 3.3. 



3.5 Cross-verification of Generality of Adapted Locomotion Gaits 

The anticipated practical implications of the analogy between the emergent properties of the 
sidewinding gaits, adapted to different fitness landscapes, are related to the possibility to 
develop a general locomotion gait which could be autonomously activated in case of any 
degradation of velocity of Snakebot. This activation could be done without the necessity for 
the Snakebot to diagnose the underlying reason for such degradation (e.g., either a 
challenging environment or degraded mechanical abilities). To verify the feasibility of such 
an approach, we examined the performance of the same three categories of best-performing 
Snakebots - (i) evolved in a smooth environment, (ii) adapted to challenging environment, 
and (ii) adapted to degraded mechanical abilities due to damage of 8 segments (as elaborated 
earlier in Sections 3.1, 3.2 and 3.3 respectively). The performance, aggregated over 20 
best-performing Snakebots, obtained from 10 independent runs of an evolution with the 
condition Fitness>100 removed from the termination criterion of GP (refer to Table 1), is 
shown in Figure 16. 
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Fig. 16. Performance of the best-performing Snakebots evolved in a smooth environment (a), 
evolved in challenging environment (b), and adapted to the degraded mechanical abilities 
due to damage of eight morphological segments (c) in various // unexpected ,/ fitness 
landscapes corresponding to a smooth environment (S), challenging terrain (C), and 
degraded mechanical abilities due to a damage to one- (Dl), two- (D2), four- (D4) and eight 
(D8) - out of 15 - morphological segments. 
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As Figure 16a illustrates, the average fitness of the Snakebots, evolved in smooth 
environment drops more than twice in challenging terrain and to 70%, 55%, 45% and 10% of 
the initial value for Snakebots with one, two, four and eight damaged segments, respectively, 
indicating relatively poor generality of these locomotion gaits. Conversely, the average 
fitness of the Snakebots, evolved in challenging terrain (Figure 16b) increases to 116% of its 
initial value in smooth terrain, and drops only to 97%, 75%, and 60% of the initial value for 
Snakebots with one-, two- and four- damaged segments, respectively. However, the average 
fitness of the Snakebots with eight damaged segments is only 25% of the initial value, 
suggesting that the degradation of the performance, inflicted by such damage is so heavy 
that it requires a specially adapted locomotion gait. The performance of Snakebots adapted 
to degraded mechanical abilities due to damage of eight segments, shown in Figure 16c, 
support this conclusion. Indeed, the average fitness of the heavily damaged (with eight 
broken segments) specially adapted Snakebots, is more than twice as high as the equally 
damaged Snakebots obtained from evolution in a challenging environment (Figure 16c). For 
Snakebots with one-, two- and four damaged segments these locomotion gaits are slightly 
superior to the gaits obtained from evolution in challenging terrain, and, naturally, 
somewhat inferior to them in a challenging environment. 

As Figure 16 indicates, not only that the adaptations to a challenging environment and 
degradation are general and robust, but these adapted gaits have a higher fitness in the 
smooth environment. However, because the energy efficiency of the adapted gaits (due to 
the additional elevation of the segments of Snakebot) is lower than the gaits evolved in a 
smooth terrain, we assume that the Snakebot generally utilizes the partially inferior (in terms 
of velocity) gaits evolved in a smooth environment and switches to the more general and 
robust gates only when difficulties are encountered. 

4. Conclusion 

We considered the adaptation of evolved locomotion gaits of a simulated snake-like robot 
(Snakebot) to two distinct changes in the fitness landscape which, in the real-world, are most 
likely to cause a degradation of the performance of Snakebot - (i) a challenging terrain and 
(ii) a Snakebot 7 s partial mechanical damage. We focused on the generality of the locomotion 
gaits, adapted to these changes in the fitness landscape, and observed the emergence of an 
additional elevation of the body and increased winding angle as common traits in these gaits. 
Discovering the strong correlation between the genotypes of the adapted gaits, we concluded 
that the adaptation is achieved through a drift towards adjacent niches in the genotypic space 
of the evolving Snakebots. Finally, we verified experimentally the generality of the adapted 
gaits in various fitness landscapes corresponding to a smooth environment, challenging 
terrain, and mechanical failures of one-, two-, four- and eight (out of 15) morphological 
segments. We argue that due to the generality of the adapted gaits, in response to an eventual 
degradation of its velocity, the Snakebot might only activate a general locomotion gait, 
without the need to diagnose and treat the concrete underlying reason for such degradation. 
We consider this work as a step towards building real Snakebots, which are able to perform 
robustly in difficult environment. 

Viewing the situational awareness, or situatedness (Pfeifer& Scheier, 1999) as a necessary 
condition for any intelligent autonomous artifact, in our future work we are planning to 
investigate the feasibility of incorporating sensors that allow the Snakebot to explicitly 
perceive the surrounding environment. We are especially interested in sensors that do not 



On the Analogy in the Emergent Properties of Evolved Locomotion Gaits of Simulated Snakebot 577 

compromise the robustness of the Snakebot - such as, for example Golgi's tendon receptors, 
incorporated inside the potentially completely sealed Snakebot. 
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1. Introduction 

At present a large number of high-rise buildings with curtain glass walls are emerging in modern 
cities. Cleaning the outer surface of these buildings is dangerous and laborious work in mid-air. 
Currently, most buildings are still cleaned manually with the help of a Gondola system or even a 
simple tool. The number of buildings with complicated shapes is increasing worldwide. Even 
skilled workers with safety ropes have difficulties in climbing those buildings. The development 
of walking and climbing robots offers a novel solution to the above-mentioned problems. Because 
of the current lack of uniform building structures, wall cleaning and maintenance of high-rise 
buildings is becoming one of the most appropriate fields for robotization (Zhang, H. et al., 2004). 
In the last two decades, several different prototypes were designed for this purpose (Sattar, T. P., et 
al., 2003), (Briones, L. et al., 1994), (Luk, B. L. et al., 2001), (Liu, S. et al., 2000). The development and 
application of cleaning robotic systems can make the automatic cleaning of high-rise buildings 
possible and thus relieve workers of this hazardous task; furthermore it can improve the level of 
technology in building maintenance. Although several robots have already been developed for 
wall cleaning, most of them can only deal with uniformly shaped planes. We have been 
developing a family of autonomous Sky Cleaner climbing robots with sliding frames for glass-wall 
cleaning since 1996. The first model (Zhang, H. et al., 2004) has only limited dexterity and cannot 
work on a vertical wall. Because it lacks a waist joint, the robot is unable to correct the direction of 
its motion. As it takes the system a long time to deal with and cross obstacles, its cleaning 
efficiency is only 37.5 m 2 / hour. The second robot (Zhang, H. et al., 2004) is easily portable and has 
a cleaning efficiency of about 75 m 2 / hour. But the stiffness of the construction is very low, causing 
a small distortion during cleaning and climbing activities. Sky Cleaner3 is a commercial product 
which is driven by pneumatic cylinders and attaches to glass walls with vacuum suckers (Zhang, 
H. et al., 2005). However, this prototype cannot be used on any other type of walls. 
SIRIUSc (Elkmann, N. et al., 2002) is a walking robot for the automatic cleaning of tall 
buildings and skyscrapers. This robot can be used on the majority of vertical and steeply 
inclined structured surfaces and facades. However, it cannot move sideways on its own and 
has to be positioned this way by a trolley on the top of the roof. Another robot was 
developed for the Leipzig Trade Fair in 1997 (Elkmann, N. et al., 2002). It was the first facade 
cleaning robot for vaulted buildings worldwide. Because of this vaulted construction of the 
surfaces to be cleaned, the robot is a very specialized system and not designed modularly. 
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In this chapter, based on analyzing the characteristics of the working target, a new kind of auto- 
climbing robot is proposed, which is used for cleaning the spherical surface of the National Grand 
Theatre in China. The robot's mechanism and its unique aspects are presented in detail. A 
distributed controller based on a CAN bus is designed to meet the requirements of controlling the 
robot. The control system is divided into six parts, five CAN bus control nodes and a remote 
controller, which are designed and established based mainly on the microprocessor P80C592. 
Then the motion functions are described in detail. From the safety point of view, the process of 
climbing from one strip to another is very dangerous because the robot has to adapt to the outer 
shape of the building. Additionally, this robot has to be quite big in order to realize all necessary 
functions. Therefore, in designing this mechanism and its controls, an equilibrium between safety 
and size has to be reached. This is why the kinematics model of the climbing process is introduced. 
For system design and control purposes, the dynamics of the robot are calculated by applying the 
Lagrange equation. The force distribution of the front and rear supporting mechanisms is 
computed in a way that ensures the safety of the climbing process. After that, a new approach to 
path planning for wall-cleaning robots is presented considering movement security, cleaning 
efficiency and the percentage of cleaning coverage. The successful on-site test confirms the 
principles described above and the robot's ability to work on a spherical surface. 



2. Overview of the robot system 

2.1 Cleaning Target 

An auto-climbing robot for cleaning glass walls with a complicated curved surface was developed 
in 2002 (Zhang, H., 2003). Taking the National Grand Theatre of China as the operation target, the 
robot can autonomously climb and clean its covering outer walls shaped in a half-ellipsoid. The 
theatre is 54 m high, and the long axis and short axis of the ellipse on the ground are 212 m and 
146 m respectively, as shown in Figure 1. The surface walls consist of more than 20,000 glass and 
titanium planks. The total outer surface covers 36,000 m 2 , with 6,000 m 2 of transparent glass walls, 
and 30,000 m 2 of Titanium walls (Liu, R. et al., 2003). The wall consists of 54 strips altogether, with 
each strip having a different height and sloping angle, as shown in Figure 2. The outer surface 
holds some LED-lamps, decorating belts and tracks. A track which was originally designed for the 
needs of construction and maintenance circles the half -ellipsoid along the rim between every two 
strips. In order to achieve a half-ellipsoid, each plank is constructed at a different size and is 
connected to its surrounding planks at an angle. 




Fig. 1. National Grand Theatre of China. 
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Fig. 2 Features of the working target. 



2.2 Mechanical design 

Based on the features of the construction mentioned above, an auto-climbing robot is designed as 
shown in Figure 3. The robot independently climbs and descends in the vertical direction and 
cleans in the horizontal direction. It takes the circling tracks as supports for climbing up and 
down between strips and moving horizontally along one strip around the ellipsoid. Its body 
consists of the climbing mechanism, the moving mechanism, two cleaning brushes and the 
supporting mechanisms. The robot is 3 meters long, 1.5 meters wide and 0.4 meter high. In order 
to keep the weight light while maintaining a dexterous movement mechanism, considerable 
stress is laid on a light yet stiff construction. Most of the mechanical parts are designed 
specifically for the robot and mainly manufactured in aluminium. 

1 




ii 



Fig. 3 Mechanical construction of the robot. 




Fig. 4 Degree of freedom. 
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The degree of freedom of the robot is demonstrated in Figure 4. There are 25 joints 
altogether, 17 out of which are active and are actuated by respective DC motors. The joints 
are categorized and described as follows: 

1 Linear active motion between the main frame and the auxiliary frame (P9); 

2. Linear active motion of two rod-clutches on the main frame (PI, P2); 

3. Linear active motion of two rod-clutches on the auxiliary frame (P8, P10); 

4. Linear active motion of the front and rear supporting mechanisms (P6, P15), and 
passive rotation of wheels on the front and rear supporting mechanisms (R5, RIO); 

5. Three rotary joints (Rl, R2, R3) and three prismatic joints (P3, P4, P5) on the front 
driving-wheel set. 

Rl represents the active rotation of the driving wheel; 

R2 is the passive rotation of the wheel paired to the driving wheel; 

R3 is a passive rotation which is used for the wheels to adapt the angle to the track; 

P3 is a linear passive motion used for the front wheel set to adapt its relative position 

to the track; 
P4 is the linear active motion to lift or lower the wheels; 
P5 is used to clamp the track between the wheels. 

6. The joints on the rear wheel set are similar to those of the front wheel set. 

7. Linear active motion of the left and right brush-sets (P7, P14), and active rotation for 
the left and the right brushes (R4, R5). 

2.3 Mechanical realization 

The main body of the robot is made up of two frames: the main frame and the auxiliary 
frame. Other functional parts are all mounted on these two frames. The auxiliary frame can 
slide along the main frame. This linear movement is actuated by the DC motor and the belt, 
both of which are mounted on the top of the main frame (shown in Figure 5). 





Bell HirriiiuiiJi' rcduwr DC ill tot Uniki- 

Fig. 5. The climbing mechanism. 

The auxiliary frame has a shape like an // n ,/ encircling the main frame to ensure safe and reliable 
movement (shown in Figure 6). A special sliding bar made of polyethylene is designed to create a 
sliding-mate between the two frames thus saving the weight of the linear guidance which is 
usually used for this kind of structure. Two rotating brushes mounted on the auxiliary frame 
move up and down and rotate for cleaning when the robot moves sideways along the tracks. 
There are two pairs of clutches (shown in Figure 7) on the main frame and the auxiliary frame 
respectively. Their function is to grasp the sliding-rods which safely slide along the track without 
becoming detached. The sliding-rods are designed to be the medium between the track and the 
robot in order to avoid the safety problem caused by the robot directly grasping the track. A DC 
motor is embedded in the mechanical clutch which allows for a very slight structure. Three 
switches on the top, in the middle and at the bottom are used to feedback its vertical movement. 
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Fig. 6. Sliding-mate between two frames. 





Fig. 7. The clutch mechanism. 

A front supporting mechanism and a rear supporting mechanism with the same structure as 
the clutches are used to adjust the orientation of the robot and support the body on the 
surface. There are two supporting wheels on the tip of the mechanism in order to increase 
the area of interaction and avoid damaging the surface (shown in Figure 8). The wheels give 
a passive turning motion when the robot climbs in the vertical direction. 




Fig. 8. The supporting mechanism. 
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A front and a rear wheel set which are used to provide the sideways driving force for the robot 
when they are clamped on the tracks are also installed on the axis of the main frame. The rear 
wheel set can move passively along the middle axis of the main frame in order to suit the various 
sizes of the planks caused by the arc shape of the building surface (as shown in Figure 9). A linear 
guidance connects the whole set to the main frame, which permits a passive sliding. The lifting 
DC motor can lift and lower the whole set to adapt to the height of the track. 

Mdiuf 





f.lnrirf:ijUiu?- 



Fig. 9. Mechanical structure of the wheel set. 

When the robot is working on such a half-ellipsoid, obstacles and the friction make it almost 
impossible to attach a safety cable to the robot from the theatre top. As a result, a special 
mechanical structure named the abdominal plate was developed and designed to solve the 
safety problem. While a set of clutches hold the sliding-rod to avoid falling down, the 
abdominal plate on the robot is interposed between two little wheels on the sliding-rod to 
avoid capsizing out of the wall (shown in Figure 10). 
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Fig. 10. Mechanical structure of the abdominal plate. 
Table 1 summarizes the main specifications of the robot. 



Body Mass (kg) 


100 


Body Mass (mm 3 ): Length xWidth xHeight 


2800 xl384x360 


Cross obstacles (mm 2 ): Height x Width 


150x100 


Max climbing velocity (mm/s) 


50 


Max working height (m) 


50 


Cleaning efficiency (m 2 / hour) 


>100 



Table. 1. Specifications of the robot. 
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3. Design of the distributed control system 

3.1 Distributed Control System Based on CAN Bus 

Five key issues are essential for this big and complicated robotic control system: a) functionality, b) 
safety c) extensibility, d) easy handling, e) and low cost. A distributed control system based on 
CAN bus is adopted to satisfy these issues, as shown in Figure 11. The system is divided into 6 
parts, five CAN bus control nodes and a remote controller (Zhang, H. et al., 2005). 
The PCM-9575 industrial PC (IPC) is the core part of the control system. It is a new EBX 
form factor 5.25" single board computer (SBC) with an onboard VIA Embedded low power 
Ezra 800 MHz. The VIA Eden processor uses advanced 0.13m- CMOS technology with 128KB 
LI cache memory and 64KB L2 cache memory. This board can operate without a fan at 
temperatures up to 60° C and typically consumes fewer than 14 Watts while supporting 
numerous peripherals. This SBC includes a 4X AGP controller, a PCI audio interface, a PCI 
Ethernet interface, and 2 channel LVDS interfaces. Its design is based on the EBX form factor 
that supports the PC/104-Plus interface for ISA/ PCI module upgrades. Other onboard 
features include a PCI slot, an LPT, 2 USBs, IrDA and 4 serial ports. 
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Fig. 11. Control system. 



A PCM3680 card is used as a communication interface with PC/104-Plus interface for the 
main node and the other CAN nodes. The main computer node is a higher-level controller 
and does not take part in joint motion control. The responsibilities of the main computer 
node include receiving orders from the remote controller, planning operational processes, 
receiving feedback information from other nodes, and giving orders to other nodes. The 
other four lower-level nodes are responsible for receiving orders from the main computer 
node and directly controlling respective joint motors. 

It is important for the control system to realize precise position control when the robot moves 
vertically from one strip of planks to another. On the other hand, some joints such as brushes and 
clutches only need an ordinary on-off control mode. As a result, two kinds of CAN nodes, both 
designed by us and mainly based on the P80C592 micro-chip, are included in the control system in 
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order to meet the requirements of functionality, extensibility and low cost. Each node is in charge 
of special functions in which all the related sensor signals are included. 

For example, the supporting mechanism node can count the pulse signals from the encoder, 
deal with the signals from the touchable sensors and other magnetic sensors, and directly 
drive the DC motors in the front and rear supporting mechanisms. To make the control 
node extensible, it contains enough I/O resources which allow for the easy attachment of 
sensors and sensor processing equipments. At the same time multiple process programming 
capability is guaranteed by the principle of CAN bus. 

Controlling and monitoring the robot is achieved through the digitalized CCD camera and a 
wireless graphical user interface (GUI) which allows for an effective and friendly operation of the 
robot. Once the global task commands are entered by the user, the robot will keep itself attached to 
and move on the surface while accomplishing its cleaning tasks. All information from its activities 
will be sent back to and displayed on the GUI during the phase of the feedback. In order to check 
out the rationality and safety of the task commands, they first have to be remembered. The 
following part includes command interpretation, task-level scheduling, and motion planning. 



3.2 Sensor System 

Multiple sensing and control systems are incorporated to handle uncertainties in the complex 
environment and realize intelligent climbing and cleaning motions. Software should be 
dexterous enough to identify the various geometries of the wall and intelligent enough to 
autonomously reconstruct the environment. The robot's body does not interact with the 
planks due to its particular mechanical structure. The tracks and the sliding-rods on the 
building surface are two important targets to be detected when the robot climbs and cleans. 
All the sensors on the robot, which can be divided into external and internal sensors, are shown in 
Figure 12. The internal sensors reflect the self -status of the robot. There are 48 limit switches in total. 
For each active joint, magnetic limit switches give the controller the position of the joint. In order to 
increase the detection diameter, a magnetic metal slice is placed at a specific position. The test 
results show that the reliability of the sensor signal is guaranteed and the diameter for detection is 0- 
3mm. For the climbing movement mechanism and the supporting mechanisms where the accurate 
position is needed, the HEDL55 optical encoders from MAXON Company are used. The control 
node can directly count the pulse signals from the encoder using the interruption. 
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Fig. 12. Sensor system. 
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The external sensors are responsible for collecting information about the operational 
environment. All of them are very important for work safety. There are three ultrasonic 
analog sensors placed at the front, in the middle and in the rear parts respectively. First and 
foremost they detect the sliding-rod in the moving direction. The accuracy of the sensors' 
output is affected by three issues including the temperature, the distance for detection, and 
the reflection performance of the objects. The US25 from TAKEX Company is chosen for our 
control system. The distance for detection is 5-25 mm; the precision for distinguishing is 40 
mV/mm; the velocity for responding is 50 ms and the operating temperature is -10 - 55 
degrees Celsius. The output of the ultrasonic analog sensor when detecting sliding-rods on 
the plank is shown in Figure 13. 
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Fig. 13. Experiment result of the ultrasonic sensor. 
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Fig. 14. Principles of three external sensors. 



588 



Mobile Robots, Towards New Applications 



There is an outshoot if the sensor detects something above the plank. On the other hand, any two 
signals can be used to compute the angle between the robot frame and the plank. To obtain precise 
data, the software will firstly choose the data and get rid of the mistakes, and then use the middle 
filter and the average filter for correcting the data, yielding accurate data for computing. 
After the position of the sliding-rod is established during the climbing movement, the robot main 
frame will pitch down and make contact with the sliding-rod controlled by the supporting 
mechanisms. We designed a kind of mechanical touchable sensor for detecting the interaction 
between the main frame and the sliding-rods. Figure 14(a) depicts its structure, for which a 
mechanical parallelogram was used. When the bottom of the main frame touches the sliding-rod, 
the contact force deforms the parallelogram, whereon the magnetic sensor sends a signal to the 
controller as soon as it has detected the metal object inserted for detection purpose. 
Meanwhile, the magnetic sensors are used for monitoring how safe the connection between 
the abdominal plate and the sliding-rod is (shown in Figure 14(b)). When the robot is 
working, there should be at least one magnetic sensor available for detecting the abdominal 
plate inserted between two little wheels on the sliding-rod. 

There are two touchable sensors on the tip of the front and the rear supporting mechanisms, 
which are used to monitor the interaction condition of these supporting mechanisms and to 
determine whether the attachment to the surface is stable (shown in Figure 14(c)). 










m 1 1. 

Fig. 15. The process of grasping a sliding-rod. 
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Some magnetic sensors are mounted on the clutches to ensure reliability when the clutches 
set out to grasp the sliding-rod. The process of grasping is shown in Figure 15. 

(a) The clutch is positioned over the sliding-rod in order to satisfy the following formula: 

L>D/2 ; 

(b) The clutch lowers until the middle switch on the clutch mechanism gives a signal; 

(c) The movement of the clutch towards the sliding-rod will stop when the magnetic 
sensor set off; 

(d) The clutch is lowered until the process is finished. 

4. Movement realization 

Movement realization includes two parts: climbing movement and cleaning movement. The 
robot can climb in the vertical direction and clean in the horizontal direction. 



4.1 Climbing movement 

Figure 16 shows the process of climbing from one layer to the next. In this process the brush 
sets are all in the up-state (no contact with the plank). 

(a) The robot is in its home state: clutches on the main and auxiliary frame are all in the 
down-state to hold the sliding-rod on track 1 and on track 2 respectively; the 
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abdominal plate is inserted in both sliding-rods; the two supporting mechanisms are 
all in the up-state. 

(b) Clutches on the auxiliary frame are raised, and the auxiliary frame is pulled up along 
the main frame. 

(c) The motion of the auxiliary frame stops when the clutches on it are right above track 
2. Then the clutches are lowered to grip the sliding-rod. 

(d) Clutches on the main frame are raised; the front supporting mechanism touches 
down on plank 1. 

(e) The main frame moves up. When the abdominal plate is out of the sliding-rod on 
track 1 and the rear supporting mechanism is above plank 1, the main frame stops 
moving and the rear supporting mechanism touches down on plank 1. 

(f) The front supporting mechanism is raised, and the main frame continues to move up. 

When the front supporting mechanism is above plank 2, it touches down on plank 2. 

(g) The upward movement of the main frame does not stop until track 3 is detected by 
the front ultrasonic sensor. Then the front or rear supporting mechanisms are 
adjusting to position the robot parallel to plank 2. Afterwards, the main frame moves 
up to insert the abdominal plate into the sliding-rod on track 3. 
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Fig. 16. Plan of the climbing motion. 
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(h) The motion of the main frame stops when the clutches on it are right above track 3. Then 
the clutches are lowered to grip the sliding-rod on track 3. The robot is now in its home state 
again, and the process of climbing up to the next strip is over. 



4.2 Cleaning movement 

Figure 17 demonstrates the process of the cleaning movement within one strip. 
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Fig. 17. Cleaning movement. 
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(a) The robot is in its home state. 

(b) The clutches on the main frame are lifted. The main frame moves up until track 1 is 
right under the rear driving-wheel set. Then the rear wheel set is lowered and the 
driving wheels are clamped onto track 1. 

(c) The main frame then moves down until track 2 is right under the front wheel set. 
Then the front wheel set is lowered and its driving wheels are clamped onto track2. 

(d) Because the clutches on the main frame are in line with the front wheel set, the 
clutches can be put down to hold the sliding-rod on track 2. Then lift the clutches on 
the auxiliary frame. 

(e) The auxiliary frame moves up until the upper edge of the brush-set is in line with 
track 2. Then the brush-set is lowered till it touches the plank. After the front and 
rear driving wheels are actuated simultaneously the robot can move to the right to 
clean the upper half of that strip. 

(f) After a circle of one half-strip has been cleaned, the brush-set moves down to the 
lower half of that strip, and the robot moves left to clean that half. 

(g) When the cleaning of that strip is finished, the brush-sets are lifted. Then both 
driving-wheel sets unclamp from the tracks and are lifted. 

(h) After that, the clutches on the auxiliary frame move down with the frame to track 1, 
and are lowered to grasp the sliding-rod on track 1. Now the robot is back in its 
home state again. 
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5. Climbing dynamics 

5.1 Kinematics analysis 

The robot autonomously climbs and cleans the elliptic surface. From the safety point of view, the 
process of climbing from one strip to another is very dangerous because the robot has to adapt to 
the shape of the building. Additionally, this robot has to be quite large in order to realize all 
necessary functions. Therefore, in designing the mechanism of this system and its controls, an 
equilibrium between safety and size has to be reached. In this section, the climbing dynamics of 
the robot are calculated based on the Lagrange formulation. We only discuss the process of 
ascending because the process of descending is similar. 

There are two phases during the process of ascending. First the auxiliary frame moves along 
the main frame, then the main frame ascends. The first part is very safe due to the clutches 
holding the sliding-rod and the abdominal plate inserted into two sets of sliding-rods. We 
will only discuss the second part in detail. 

The climbing dynamics of the robot are analyzed by the application of the Lagrange 
equation, described as (1) (Fu, K.S. et al., 1987). 

at dq ■ dq 

Where F, is the generalized forces; L is the Lagrange function, as shown in (2). 

L = T-V (2) 

Where T is the kinetic energy of the system; and V is the potential energy. We can change (1) 
into the following format (3). 

M(q)q + C(q,q)q + N(q,q) = F (3) 

Where F is the generalized driving force; q is the generalized coordinate; C(q,q)q described 

with (4) is decided by the factors of the Coriolis force and the centrifugal force; N(q,q) is 

decided by the nonconservative and conservative forces. Furthermore it can be described as 
(5) when the friction is neglected. 

c ^4ZJ^+^-^k (4) 

2 £i [ dqk dq J dqi \ 

Nt(q,4)~ ( 5 ) 

oq t 

The kinematics model of the robot during climbing is shown in Figure 18. Where oxyz is the 

world coordinate and the xoy plane is paralleled with the lower plank; and o'x'y'z' is the 

reference coordinate fixed at the robot's body, with the x' axis along the width of the frame, 

the y' axis along the length and the z axis pointing vertically upward from the main frame. 

The two points of o and o' are superimposed. 

In Figure 18 below, the parameters are shown (Zhang, H. et al., 2005). 

t3 is the vector from the centroid of the auxiliary frame to the centroid of the mainframe; 

f4 is the displacement of the front supporting mechanism; 

r$ is the displacement of the rear supporting mechanism; 

j3 is the rotating angle of the mainframe according to the x axis, it is also named d x ; 
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(p is the rotating angle of the mainframe according to the z' axis; 

a is the angle between the lower plank and the gravity vector; 

R is the length of the clutches; 

Li is the distance between the front supporting mechanism and the centroid of the 

mainframe; 

L2 is the distance between the rear supporting mechanism and the centroid of the 

mainframe; 

mi is the mass of the mainframe; 

m2 is the mass of the auxiliary frame; 

Jicx/ Jicy and Ji cz are the moments of inertia of the mainframe; 

J2cx , J2cy and J2cz are the moments of inertia of the auxiliary frame; 

F r 3 is the driving force for climbing; 

Ffb is the driving force for the front supporting mechanism; 

Fbb is the driving force for the rear supporting mechanism; 

M is the driving moment generated by the supporting mechanisms. 




Fig. 18. Kinematics model of the climbing process. 



When the front supporting mechanism is working, it is described as (6); when not, it is 
described as (7). 



M = F FB x(L l +r 3 ) 
M = F BB x(L 2 -r 3 ) 



(6) 
(?) 



While climbing up, the front and rear wheel sets do not work so that the freedom in the x 
direction is not available. The freedom of climbing is three and can be described with the 
generalized coordinate q (r^,(p,j5 ). There are two situations that should be avoided during 
this process. One is the non-brace that means neither the front supporting mechanism nor 
the rear one works; the other is the excessive-brace that means two supporting mechanisms 
work together. There are some constraints between u, rs and p , as shown in (8) and (9). 



r A ={r 3 +L x )p 



(8) 
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r 5 =HL 2 -r 3 )fi (9) 

According to (2), the kinetic energy of the mainframe is Ti, as shown in (10). 

(10) 



T\ = -™\V\c 2 +-J\cz<P 2 +~J\cxP 2 



The absolute velocity of the mainframe Vi c is described as (11). The following velocity Vi e and 
the relative velocity vi r are shown in (12), (13). 



v \c = v \e+ v \r 






(11) 

(12) 
(13) 



As shown in Figure 19, the vector r 3 is orthogonal withr 3 #>, if v le and v lr are projected 
along x'y'z' coordinate axes; we can get (14). 




Fig. 19. Kinematics model projected on the x'y'z' coordinate axes. 
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(14) 



2 + r 2 



v lz , = ^R 2 + ri 2 ]3 ■ ri = r 3 



The kinetic energy of the mainframe is shown in (15). 

Tl = \rn x [^ -RPf +{r,q>) 2 +(r 3j 3) 2 ] + ^J lcz <p 2 +^J lc J 2 



(15) 



Since the zero potential energy position is on the horizontal plane passing though point o, 
the potential energy is described as (16). 



V\ - m \gR sin(or - p) + migr 3 cos(a - fi) 



(16) 



594 



Mobile Robots, Towards New Applications 



In the same way, the kinetic energy and the potential energy of the auxiliary frame are 
calculated in the following (17), (18). 



T 2 =-m 2 (Rj3) 2 +-J 2c J 2 
V 2 = m 2 gRsin(a-p) 
In (19), all these results are inserted into the Lagrange function. 

L = -m x r 2 + —m x R 2 j3 2 - m x Rr z j$ + -m x r 2 (p 2 + -m x r 2 j3 2 + -m 2 R 2 j3 2 

? 2 , 1 



+ ~J% 



1 r -2 1 T 
+ - J \cz<P + - J \cx) 

2 2 z 
- m x gR sin(a - P)- m x gr 3 cos(or -fi)- m 2 gR sin(or - /?) 



(17) 
(18) 



(19) 



According to the Lagrange equation (20), the dynamic characteristics of the climbing 
movement are reached, as shown in (21) -(23). 



___(_____) ______ = F 

dt 3r 3 3r 3 

i.(^)_aL = o 

dt d(p d(p 
dt d/3 dj3 



(20) 



m { r 3 - m x Rfi — m^qt 2 - m { r 3 p + m l g cos(a — J3) = F r3 

2 
{rn\r^ + J Xcz )(p + 2m x r 3 r 3 (p = § 

-m x Rr 3 +(m x R +m 2 R +m\r^ + J Xcx + J lcx )P + 2m x r 3 r 3 p 
-n\gRcos((x- j3)+n\gj§sin(icx- j3)-m2gRcos((x- fi) = M 



(21) 
(22) 

(23) 



We can change the dynamic characteristics into format (3), as shown in (24). All of the 
results are important for system design and the design of the controlling mechanism. 



M(q,q)-- 








m l r 3 2 +J u 



-m x R 




-m x R m x R 2 +m 2 R 2 + m l r 3) 2 +J\ cx +J 2cx 



C(q,q)-- 



N(q,q)-- 



-m x r 3 (p — m\r 3 P 
m x r 3 (p m x r 3 r 3 

m x r 3 p ^i^3^3 

m x g cos(a - p) 




F r 


M 



- m x gR cos^ - p) + m x gr 3 sin(« - p)-m 2 gR cos(a - p) 



(24) 



5.2 The pitching forces 

There are two kinds of dangerous cases during the climbing process, one is falling down 
and the other is capsizing out of the wall, as shown in Figure 20. The climbing robot has to 
be safely attached to the wall surface and has to overcome its gravity. The attachment plane 
should adapt to the complicated shape of the building. That is the first difference between a 
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wall climbing robot and an ordinary robot walking on the ground. 

As mentioned before, the supporting mechanisms are used to adjust the alignment of 
the robot and to support the body on the surface. In order to meet the requirements of 
safety, the touching pressure between the supporting mechanisms and the surface 
should be sufficient. That means the interaction conditions on the surface are stable. 
On the other hand it is also quite dangerous if the force is too great as this could 
crush the planks. As Nu m i t (1500N) is the bearing capability of the plank, at the 
beginning of the designing process, we should make sure the touching pressure is 
smaller than this limitation. 




Fig. 20. Dangerous cases during the climbing process. 



The front supporting force is shown in (25). Where L is the length of the mainframe; I is the 
negative of r$ in order to compute the distribution of the force. 

G 2 ■ R cos a + G x ■ (R cosa + l sin a) n ^\ 

F ™ = r^ (25) 

When all the conditions shown below apply, R = 150mm, L = 2800mm, Gi = 60kg, G2 = 40kg, 

Li = 200mm, L2 = L/2=1400mm, a = 0-90°, the force distribution is gotten, as shown in Figure 
21(a). Here the range of I should be within [-1000,700] mm due to mechanical constrains. 
According to the same process, the distribution of the rear force is shown in Figure 21(b). 
From these two Figures, we can see the maximum of the supporting forces meet the requests, 
as shown in (26). 

FBBmax = 37.5Kg < F FBm ax = 88.6Kg < Numit (26) 
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Fig. 21. The front and rear supporting pressure distributions. 



6. Cleaning Trajectory 

6.1 Characteristics of path planning for wall cleaning robots 

Another important issue for a wall-cleaning robot is path planning. A specific cleaning 
trajectory is essential for the cleaning movement to cover all the unoccupied areas in the 
environment. A lot of research has been devoted to the path planning problem, such as the 
grid-based method (Zelinsky, A., et al., 1993), the template-based method (Neumann de 
Carvalho, R. et al., 1997) and the neural network approach (Yang, S.H. et al., 2004). But none 
of these path planning methods can be directly used for the wall cleaning robot due to the 
characteristics of its special working environment. 

Area-covering operation is a common and useful kind of path planning also named 
complete coverage path planning (CCPP), which requires the robot path to cover every part 
of the workspace (Hofner, C, et al, 1995) (Pirzadeh, A., et al, 1990). The CCPP is suitable for 
wall-cleaning robots due to robotic operational characteristics. This project is intended to 
develop an effective and easy-to-compute path planning for robotic cleaning systems on 
high-rise buildings. Even if the path planning methods for mobile ground-robots and those 
of wall-cleaning robots have a lot in common, the latter still feature three particular 
characteristics (Zhang, H., et al., 2005): 

1) Prior knowledge of the global environment and local unexpected obstacles; the global 
environment is known in advance so that it can be considered as a static model. On the other 
hand, the work space includes numerous unexpected obstacles such as window frames and 
bars. From this point of view, the local work environment is dynamic and unknown. The 
cleaning trajectory is also based on local sensorial information. 

2) Work environment constraints and cleaning function constraints; the robot should move 
in both the vertical direction as well as the horizontal direction to get to every point on the 
cleaning surface. In order to effect the cleaning, the robot has to face all obstacles and cross 
them safely and quickly so that the cleaning movement covers the whole area. At the same 
time, the climbing robot has to be safely attached to the glass wall and has to overcome 
gravity. The cleaning trajectory has to take both aspects into account. 

3) Evaluation by synthesis of standards (Zhang, H., et al., 2005); the standards of choice and 
evaluation for the different kinds of path planning should synthesize work safety, cleaning 
efficiency and the percentage of cleaning coverage. The safety is the first important factor for 
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evaluation. But even if it is very reliable, the cleaning robot is still not a satisfactory product 
if it takes longer to clean a certain area than human workers would. The robot should find 
an efficient cleaning trajectory to carry out its work. At the same time, the percentage of 
cleaning coverage is also quite important from the practical point of view. It would be 
unacceptable for the robotic system to only achieve the cleaning of selected parts of the 
target surface while human workers in a gondola can clean it almost 100%. 



6.2 Subdividing the work target 

Path planning and the path tracking strategies for mobile robots are highly dependent on 
the work target characteristics. This project works with the concept of the limitation of the 
work area, so that the constructional consistency, continuity and accessibility are satisfied 
within any subdivided area. Consistency means the curtain-wall has the same structure 
within one area; continuity means that the area is a plane or a curve with a sufficiently large 
radius; and accessibility means that any spot on the wall can be reached. There are three 
guidelines for subdivision. 

1) Boundary rule; on any work space there must be real horizontal and vertical boundaries 
which are natural marks for subdivision. 

2) Medium rule; the work target can be subdivided into different parts according to its material. 
Generally, different methods and techniques for cleaning will be used on different planks. 

3) Obstacle rule; there are many kinds of obstacles on the work space. The obstacles can be 
divided into four kinds: unexpected and expected obstacles according to their positions; 
dangerous and safe obstacles according to their criticality to the robotic system. Some 
obstacles are too high or too big for the robot to cross. They should be considered as a 
special, dangerous kind of boundary to be avoided. 

Using these three guidelines, the global workspace can be subdivided into several regions, 
as shown in Figure 22. We are dealing with six work space regions. The material of Region I 
and IV is glass; other areas are Titanium. 

Cleaning and walking are tracked by synchronizing the global information and the local 
perceived situation in the phase of path planning. Firstly the robot should choose a direction 
along which the cleaning work can be carried out. Here a representative subdivided area on 
the border between Region I and Region II, as shown in Figure 23, is the work target based 
on the principles of area definition. 




(a) The draft of the building construction. 
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(b) Regions definition from the top view. 
Fig. 22. Results of subdivision. 

If the robot cleans the work space in the vertical direction, it has to cross the obstacles 
several times. It is much more difficult to clean the work target due to numerous horizontal 
obstacles. But the robot can clean uninhibited in the horizontal direction, because that way 
the wall is considered as forming a plane. The robot begins to clean the wall from the upper 
left point. It will move on to the next strip when the first strip of planks is finished cleaning. 
Some areas near the border cannot be cleaned for safety reasons. The coverage percentage 
on this small work area is over 95%. This value will be higher from the global point of view. 
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Fig. 23. Cleaning trajectory. 

7. Conclusion 

In contrast to conventional theoretical research, this project finishes the following innovative work: 
1. This paper described a new kind of auto-climbing robot used for cleaning the complex 
curved surface of the National Grand Theatre in China. A special movement mechanism 
was developed and designed to satisfy weight and dexterity requirements. The robot has 
been tested on a demo wall which was built based on real building design. Figure 24 
shows the robot climbing up in the same process as Figure 16. The robot can climb up and 
down walls reliably with an average climbing speed of 200mm/ s, a sideways moving 
speed of 100mm/ s, and has a cleaning efficiency of 150m 2 /hour. These tests have proven 
the feasibility of the mechanical design and control system of the robot. 
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Fig. 24. Real climbing experiment. 
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4. 



An intelligent control system based on CAN bus was designed to control the robot. 

The advantages and the characteristics were analyzed. Some key issues such as 

sensor technology, which is necessary for such an autonomous robot, were studied in 

detail. 

A kinematics model of climbing was concluded. The climbing dynamics of the robot 

are stated by the application of the Lagrange equation. All of the results are 

important for system design and the design of the controlling mechanism for this 

climbing robot. 

The cleaning trajectory was determined and evaluated by the synthesis of work 

safety, cleaning efficiency and cleaning coverage percentage. Testing results with the 

robot have verified the effectiveness of the CCPP. 
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